Message Logging using JavaScript and Service Callout

Google Cloud Apigee
1

I originally added this as an answer to another similar question on this topic, but decided it was probably better to put in it’s own question…

I need to send logs to an ELK stack within my corporate back-end. I am hitting a RESTful endpoint via HTTP. But I found the MessageLogging policy very limiting in terms of the material I wanted to log and how I wanted to structure the payload.

I have built a log payload through JavaScript, which works well and contains a wealth of information in the structure I need it… but it means I can’t put it in the ClientPostFlow, as it doesn’t appear you can execute scripts in this flow.

So I have put it in the Response PostFlow instead. I have bundled the JS step and a Service Callout into a Shared Flow, and set the whole lot on “Continue on Error”. OK so far…

But… Even with the Service Callout running asynchronously, the average proxy response time has jumped from an average of 15ms to an average of 60ms. When you look at the Trace, it’s difficult to see where this extra overhead has come from.

7691-trace.png
7691-trace.png375×176 5.47 KB

If you take this Trace above, it shows the JS step taking < 1ms, and the SC step taking < 1ms. The overall Flow Callout is taking < 1ms. BUT… the “Response Sent to Client” overall is showing an elapsed time of 58ms. That doesn’t seem to tally-up with the response times of each of the steps within the flow.

I guess there’s a few questions here:

  1. Any idea where my overhead is coming from, or how I figure it out?
  2. Is there any plan to allow scripting in the ClientPostFlow, as I know this is the best place to put the logging?
  3. Would I gain any benefit from building my payload in Javascript, but then using a MessageLogging policy rather than a Service Callout policy to actually make the call? Does Message Logging do anything particularly ‘clever’ that Service Callout doesn’t do?
1 Like
2

Yes, the PostClientFlow is designed specifically for your use case. That is the best place to insert a messagelogging policy, as it will not impact the response time the client experiences.

Relevant doc.

i think that building the payload in JS and then transmitting it via MessageLogging attached within the PostClientFlow is a good approach.

As for why you experience the +50ms in response time even when you use the fire-and-forget approach with the httpClient inside the JS policy, I’m not sure. It could be that the resolution of the API Proxy is delayed even if within the JS code, you don’t wait for a response.

Inviting comment from @richarst .

And maybe @Peter Johnson

3

John - thanks for making this a new question! That’s what we had hoped people would do!

4

But MessageLogging policy does not allow logs to be sent to an HTTP / HTTPS target? Can I use another policy inside PostClientFlow?

Also to clarify, I am not using the httpClient inside a JS Policy. I am using JS policy to build the message, then Service Callout to make the call.

5

Ah, ok.

Well then.

Two suggestions:

Try to eliminate the Response element in the SC policy config.

With no response it tells the SC “do not wait”.

Alternatively, you can try using the httpClient from within the JS policy.

Again, don’t wait for the response. Just send it.

I would try one or both of those to see if it eliminates the delay.

It’s not quite as nice as the messageLogging policy in a PostClientFlow, but almost as good.

6

Yes, I already omit the Response element from the Service Callout in order to make it async.

I’ll explore the httpClient in Javascript to see if it makes any difference.

7

Do you have any examples or documentation you can point to for this type of solution, as opposed to the MessageLogging example shown above?