TL;DR
This article provides a step-by-step guide on how to leverage Apigee resources to store dynamic information you might want to retrieve as part of your API flow.
To share a possible real-life scenario I’ll use to guide you through all the steps, you can think about the integration between Apigee and a GenAI based “intent detection” layer requiring the list of agents a certain user is authorized to reach.
For the context of this article, we’ll leverage an existing Apigee X PAYG organization so we won’t explain how to get it up and running, but if you are interested in the provisioning process, please refer to the official documentation.
Context
Before starting with the technical details of the implementation, let’s deep dive a little bit on the Apigee resource concept and the use cases you might want to implement within your organization or company relying on this feature.
Apigee Resource
From the documentation
Resources are the files that implement the code or configuration to be executed by a policy when attached to an API proxy
This means you can store either pieces of code to be executed when associated with a Javascript policy but also to store dynamic information you might want to retrieve at real-time as part of your API flow.
If while reading you are thinking: “Well what’s the different between this new thing and KVM?”
Actually you are right, this is pretty much the same but Apigee KVM but has a size limit (10KB) and is not meant to be specific for storing long configuration and map objects of any format.
So at this point my follow-up question if I were in you would be:
Why would anyone need to store dynamic information in real time?
And here you are, few example of some reasons you might want to set up and use a resource in your own home set up:
- Mapping configuration: as we will see in this article, Apigee resources are pretty useful when your goal is retrieving mapping at real-time and use them as part of the API flow.
- Authz/Authn: another example would be checking if a given user/company has the necessary grants to perform an action on a backend when the out of the box implementations (API Key, OAuth, etc) are not an option
- OpenAPI Spec: Apigee resources can be also used to retrieve the latest OpenAPI specification to perform a function calling from a GenAI agent to your desired tool.
Another interesting feature? You can store files from plenty of extensions to accomodate almost all the customer’s requirements, starting from JS, to OpenAPI spec, JSON and more. If I was able to convince you, go to the next chapter to check what I have implemented levereging Apigee Resource.
How to “use” resources for dynamic mapping for an Intent Detection layer
Ok, let’s move to the hands-on section of the article: how can I leverage an environment-level resource to return back to an “intent detection” layer, the list of agents a certain user is authorized to reach.
And let’s add some more requirements to the challenge:
- I want to expose an API (let’s say /v1/backends) returning the list of agents available for a given user (user_type actually)
- The mapping between user_types and agents is provided as JSON file
- I would like to have the possibility of updating the mapping at real-time without re-deploying the API Proxy (less change as possible on Apigee)
How can I do this?
Create an environment level resource
While we have picked an environment-level resource, it’s important to note that, based on your requirement, each resource can be associated with an API Proxy directly.
To create such resource, we are leveraging the resourcefile Apigee Management API, here is the method and url detailed information:
POST /v1/organizations/<ORGID>/environments/<ENVID>/resourcefiles?name=mapping.json&type=jsc
where ORGID is your organization_id and ENVID is your environment_id.
As body of the API call, the JSON file should be passed to Apigee together with the following Header
- “Authorization: Bearer $TOKEN”
- “Content-Type: multipart/form-data”
So, if we want to use cURL, for example
curl "https://apigee.googleapis.com/v1/organizations/<ORGID>/environments/<ENVID>/resourcefiles?name=mapping.json&type=jsc" -X POST -H "Authorization: Bearer $TOKEN" -H "Content-Type: multipart/form-data" -F "file=@mapping.json"
An example of JSON mapping?
{
"user_type_1": {
"recipe_creator": {
"name": "Recipe Generator",
"description": "This is to generate food recipes",
"url": "/v1/recipe"
},
"drink_creator": {
"name": "Drink Generator",
"description": "This is to generate drink recipes",
"url": "/v1/drink"
}
},
"user_type_2": {
"recipe_creator": {
"name": "Recipe Generator",
"description": "This is to generate recipes",
"url": "/v1/recipe"
}
}
}
Configure the API Proxy to retrieve the mapping from resource
Now that Apigee is storing our JSON mapping, how do I retrieve the value in our API Proxy?
Doing that is very simple, we are just going to add an Assign Message policy to our proxy and assign the value of the mapping.json file to a flow variable (in my example, flow.mapping_json).
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<AssignMessage continueOnError="false" enabled="true" name="AM-get-mapping">
<AssignVariable>
<Name>flow.mapping_json</Name>
<ResourceURL>jsc://mapping.json</ResourceURL>
</AssignVariable>
</AssignMessage>
As you can see, the only property to set up to reference our mapping.json file stored in the resource is the ResourceURL. Pretty simple right?
Once stored in the flow.mapping_json flow variable, you are free to do anything you need with the mapping.
In my case, I have created a JS code to retrieve, based on the user_type, the proper list of backends and return it as part of the payload response.
var mapping = context.getVariable("flow.mapping_json") || undefined;
var user_type = context.getVariable("flow.user_type") || undefined;
var filtered_mapping = "None";
if(mapping) {
payload_mapping = JSON.parse(mapping);
if (payload_mapping[user_type]) {
filtered_mapping = JSON.stringify(payload_mapping[user_type]);
context.setVariable("flow.filtered_mapping", filtered_mapping);
}
if (filtered_mapping == "None") {
context.setVariable("js.error", "Invalid Mapping");
throw new Error("Failed to perform backends lookup");
}
} else {
context.setVariable("js.error", "No JSON payload to parse");
throw new Error("Failed to load mapping JSON file");
}
What if I want to update the mapping value?
Through time, the JSON mapping linked to the Apigee resource could potentially change so, how can we update the value?
As before, we are going to leverage the resourcefile Apigee Management API, there is the method and url:
PUT /v1/organizations/<ORGID>/environments/<ENVID>/resourcefiles/jsc/mapping.json
where ORGID is your organization_id and ENVID is your environment_id.
As body of the API call,the updated JSON mapping should be passed together with the following Header
- “Authorization: Bearer $TOKEN”
- “Header: Content-Type: multipart/form-data”
As before, if we are looking at a cURL example
curl "https://apigee.googleapis.com/v1/organizations/<ORGID>/environments/<ENVID>/resourcefiles/jsc/mapping.json" -X PUT -H "Authorization: Bearer $TOKEN" -H "Content-Type: multipart/form-data" -F "file=@mapping.json"
What if I want to leverage a CI/CD pipeline to do so?
In my case, instead of calling directly the Apigee Management API with the updated JSON Mapping, I have preferred to create a very simple CI/CD pipeline so every time I push the latest version of the JSON file to GitHub, the update Apigee Resource call is perfomed.
How did I do this?
- Connect the GitHub repository to Cloud Build (howto)
- Create a Cloud Build Trigger running when a “push” to the repo happens (howto)
- Update the Cloud Build configuration inline YAML file to execute a script (stored in the repo itself) executing the previously shared Apigee Management call by passing the latest version of the mapping.json file.
In case you want to just copy and paste, here is my “promote.sh” script
#!/bin/sh
export APIGEE_ORG=<ORGID>
export APIGEE_ENV=<ENVID>
export APIGEE_RESOURCE=mapping.json
if_res=$(curl "https://apigee.googleapis.com/v1/organizations/$APIGEE_ORG/environments/$APIGEE_ENV/resourcefiles/jsc/$APIGEE_RESOURCE" -H "Authorization: Bearer $(gcloud auth print-access-token)" | jq -r '.error.code')
if [[ $if_res == "404" ]];
then
echo "Resource does not exists";
else
curl "https://apigee.googleapis.com/v1/organizations/$APIGEE_ORG/environments/$APIGEE_ENV/resourcefiles/jsc/$APIGEE_RESOURCE" -X PUT -H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: multipart/form-data" -F "file=@mapping.json";
echo "Resource $APIGEE_RESOURCE successfully updated";
fi
Conclusion
That’s all folks!!
Let me know your thoughts about this article and feel free to reach out in case of any question.
This article was originally published on Medium.