Sumo Logic is a popular cloud based logging solution. They support local file upload, SysIog and HTTP Source collection types. For API logging out of Apigee, the HTTP endpoint is the way to go. The Syslog endpoint requires the installation of a “collector” component which isn’t possible with Apigee.
This solution uses a single JavaScript policy to make HTTP POST requests to Sumo Logic HTTP Source Collector.
Using the JavaScript policy to make asynchronous HTTP requests is a standard approach documented here.
Caveat: JavaScript policies cannot be used in the PostClientFlow, but that’s OK, put them in the Proxy PostFlow or wherever else you want to log, like in the DefaultFaultFlow.
Setting up logging to Sumo is really simple, here’s a step by step guide with a realistic and working JavaScript policy that shows data being logged in Sumo Logic.
Setup Sumo Logic
-
Create a free trial account at Sumo Logic.
-
Select “Set Up Streaming Data” using the setup Wizard.
- Select the data type “Your Custom App”
- Set up the collection using “HTTP Source”
- Configure the HTTP source, this can be what ever hierarchy you want.
- Save the obfuscated URL to be put into your policy (or into a configuration KVM in a real world scenario), there is no security other than the URL value itself.
Grab a cup of coffee, wait for the email from Sumo indicating your log is ready (few minutes, maybe more).
Add Logging to your Proxy
In the meantime, you can configure your API to send log entries. In this example, I use the following JavaScript policy in the Proxy Post Flow:
// Get variable which I set from a previous KVM configuration lookup.
// I use this to turn logging on or off
// var logging = context.getVariable("exco.logging");
var logging = "true";
if (logging == "true") {
// Get variable from previous KVM configuration lookup
// var logServerURL = context.getVariable("exco.loggingUrl");
var logServerURL = "https://endpoint1.collection.us2.sumologic.com/receiver/v1/http/ZaVnC4dhaV1oma90Vvb...";
// Debug
print('LOGGING ' + logServerURL);
// calculate response times for client, target and total
var request_start_time = context.getVariable('client.received.start.timestamp');
var target_start_time = context.getVariable('target.sent.start.timestamp');
var target_end_time = context.getVariable('target.received.end.timestamp');
var request_end_time = context.getVariable('system.timestamp');
var total_request_time = request_end_time-request_start_time;
var total_target_time = target_end_time-target_start_time;
var total_client_time = total_request_time-total_target_time;
// Create the log object which can be queried by field in Sumo
var logObject = {
"organization": context.getVariable("organization.name"),
"environment": context.getVariable("environment.name"),
"apiProduct": context.getVariable("apiproduct.name"),
"proxyName": context.getVariable("apiproxy.name"),
"appName": context.getVariable("developer.app.name"),
"verb": context.getVariable("request.verb"),
"url": '' + context.getVariable("client.scheme") + '://' + context.getVariable("request.header.host") + context.getVariable("request.uri"),
"responseCode": context.getVariable("message.status.code"),
"responseReason": context.getVariable("message.reason.phrase"),
"clientLatency": total_client_time,
"targetLatency": total_target_time,
"totalLatency": total_request_time
};
var headers = {
'Content-Type': 'application/json'
};
// Debug
print('LOGGING OBJECT' + JSON.stringify(logObject));
var myLoggingRequest = new Request(logServerURL, "POST", headers, JSON.stringify(logObject));
httpClient.send(myLoggingRequest);
}
Test the Solution
Now fire off a few requests and login to Sumo to search the logs, I just searched “*”, you can build more complete queries.
Create Dashboards
There are many ways to structure your logs, by environments, by APIs, etc., take some time to figure out how you want to view the logs. In a large enterprise with lots of logs in Sumo Logic, I created top level for apigee, then environments (dev, test, prod), then APIs. Then I create dashboards with various views, for example:
Happy Logging!