How to hide the Loggly customer token from logs?

Google Cloud Apigee
1

When I use the Message Logging policy to send the logs to Loggly, the Loggly customer token is also getting saved in the logs. How can I hide this from the logs?

Here is the policy that I am using:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<MessageLogging async="false" continueOnError="false" enabled="true" name="Message-Logging">
    <DisplayName>Message Logging</DisplayName>
    <Syslog>
        <Message>[xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx Hello World!]</Message>
        <Host>logs-01.loggly.com</Host>
        <Port>514</Port>
        <Protocol>TCP</Protocol>
    </Syslog>
</MessageLogging>

Corresponding log in Loggly:

[xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx Hello World!]
1 Like
2

Hi @Bibin Kurian

Could you try by changing the content of tag as below:

[xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx] Hello World!

I guess, “]” bracket should close before the message content as specified in the example:

http://apigee.com/docs/api-services/reference/message-logging-policy#thirdparty

3

Hi @rdoda

I tried this but didn’t work. The new log is as below:

[xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx] Hello World!
4

Hi @Bibin Kurian

One another aspect is pending and that is Loggly requires message to be RFC5424 compliant. ref Loggly documentation @

https://www.loggly.com/docs/streaming-syslog-without-using-files/

RFC5424 requires header and structured data before the message. Ref section 6 of the specification:

http://tools.ietf.org/html/rfc5424#page-191

Based on these requirements; could you specify the message tag as:

<14>1 - - - - - [xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx] Hello World!

Note that blank space is required between all fields except first and second. There is no blank space between “>” and “1”.

Cheers,

Rajesh Doda

5

@Bibin Kurian - Karen from Loggly here. Try moving the closing bracket before the message. Let us know if that helps!

6

@rdoda This also didn’t work.

7

@Karen Sowa I tried this, but not working.

8

Could you copy new message tag and the log content.

9

Hi @bibin_kurian,

Are you able to resolve this issue? If yes, could you share the solution. If not, could you copy the message tag and content at log.

Regards,

Rajesh

10

Hi @rdoda,

I couldn’t solve the issue yet. Here is my Message attribute and log:

<Message><14>1 [xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx] Hello World!</Message>

Log:

<14>1 [xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx] Hello World!
11

Hi @bibin_kurian,

you forget to specify 5 “-” separated by space character as mentioned in my earlier post. Note that there is a space between “1” and first “-” as well as last “-” and “[”. These are required as specified in the RFC5424 (Ref section 6).

<14>1 - - - - - [xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx] Hello World!

Give a try and let us know.

Cheers, Rajesh

1 Like
12

Hi @rdoda,

Surely the above example is malformed XML? Why is the <14> required?

Thanks,

Sean

13

Hi @rdoda,

Thanks for pointing out the mistake. But no luck with this also. Here is my new message and log:

Message:

<Message><![CDATA[<]]>14<![CDATA[>]]>1 - - - - - [xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx] Hello World!</Message>

Loggly log:

<14>Sun Aug 16 22:09:59 UTC 2015Info: <14>1 - - - - - [xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx] Hello World!

When I send the below data using curl, Loggly is properly hiding the customer token. But when I use the same data in Apigee message logging policy, the customer token is getting added to the logs.

curl -H "content-type:text/plain" -d "<14>1 2015-16-08T10:14:15.003Z - - - - [xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx] Hello World!" [http://logs-01.loggly.com/inputs/xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx/tag/http/](http://logs-01.loggly.com/inputs/xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx/tag/http/)
14

Hi @bibin_kurian,

Some points:

  1. Curl command for loggly works fine as now your message is RFC5424 compliant.

  2. Latest log specified by you has "<14>Sun Aug 16 22:09:59 UTC 2015Info: " as prefix. Was it not coming in the earlier logs? This text is missing in the log contents specified earlier.

  3. Try as below and let us know:

[xxxxx] HelloWorld!

Note: “xxxxx” are corresponding to yours Loggly private enterprise number.

Cheers, Rajesh

15

@rdoda,

Regarding second point, this prefix was there in earlier logs as well. Is this something that Apigee adding as header? If so, is this a RFC5424 compliant header?

Regarding third point, I don’t think the messages will get logged to my Loggly account without my customer token.

Thanks,

Bibin

16

Apigee’s syslog policy adds the timestamp, so as @rdoda pointed out, all you need is [xxxxx] HelloWorld!, and you will see that loggly has received:

<14>Tue Aug 18 09:59:47 UTC 2015Info: [xxxxx] HelloWorld!

To answer your original question - I have been unable to hide the customer token from the logs, and am not sure it is possible when using Apigee’s Message Logging Policy.

1 Like
17

@bibin_kurian Please open a ticket, our support team is here to help you out! http://bit.ly/1TRBZ7k - Karen

18

I can able to reproduce same, Any pointers? @rdoda @krupa @sriki77

1 Like
19

@Anil Sagar, could you please get this logged as an issue with the apigee product team?

20

@Cladius Fernando ,Sure i will do same. I will keep you posted.