Accessing a Google Cloud Platform based service using JWT and a service account

We have a GCP-based microservice (built within our company) which we are attempting to access via one of our proxies. We set up a service account within GCP which should have access to this resource. We downloaded the credentials file associated with service account (edited for security):

{ 
  "type": "service_account",
  "project_id": "cluster",
  "private_key_id": "93158289b2734d823aaeba3b1e4a48a15aaac",
  "client_email": "apigee-orderservices-dev@ourcluster.iam.gserviceaccount.com",
  "client_id": "1167082158558367844",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/apigee-orderservices-dev%40ourcluster.iam.gserviceaccount.com",
  "private_key": "-----BEGIN PRIVATE KEY-----\nMIIEvgIBADANBgkqhkiG9w0BAQE...8K5WjX\n-----END PRIVATE KEY-----\n"
} 

We are attempting to use a GenerateJWT policy, storing the private_key value in an encrypted kvm so that we can get an authentication token using this JWT request, but I am currently getting “cannot instantiate private key”. I’m pretty sure that I’m missing something basic here, but any assistance would be greatly appreciated.

1 Like

Let me look

It works for me. I tried 2 different ways.

In either case, the goal was to create a JWT that is signed by an RSA key, obtained from the google service account credentials file. The payload of that JWT should look like this:

{
  "iss" : ServiceAccount client_email,
  "scope" : scope,
  "aud": ServiceAccount token_uri, 
  "iat": nowInSeconds,
  "exp": nowInSeconds + (3 * 60)
}

In either case, the first thing I did was: create a service account and download the .json file.

Then, Option 1:

  1. I extracted the private_key string from the json file,

  2. Using a text editor, I replaced all \n characters with “newline” . The result looks like this:

    -----BEGIN PRIVATE KEY-----
    MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCsGyx52HRj4z2D
    oMdmMcdVSU1nbZ6m/r4QOsiFL/KKcdnw6lMkUxwzaIKR27IK1Hngn8SROM5UAp3p
    WuGwyF2vlIfzOdQXFrjScCT0XJu5wjCjqZe0eUhEWMtBbiKTNSb2K536EG3iy9oY
    sUZ4RjNvHaRqGn8HAr6mGM5Q8fILIqSYQwNO+htvcso5TKHcR6b79Nz9TcqC6ger
    WG5pioXNmXSuuMHkTexqbLjdP0MIub/ViiqPIiWkGtv8wAZu+3NuIatuz1VFIq+v
    ...
    -----END PRIVATE KEY-----
     
    
  3. added THAT ^^ to the Encrypted KVM via the Admin UI, stored under the client_id.

  4. Added a KVM-Get policy that looks like this:

    <KeyValueMapOperations name="KVM-Get-1" mapIdentifier="secrets">
        <ExclusiveCache>false</ExclusiveCache>
        <ExpiryTimeInSecs>300</ExpiryTimeInSecs>
        <Get assignTo="private.rsakey">
            <Key>
                <Parameter>104855500587360709513</Parameter>
            </Key>
        </Get>
        <Scope>environment</Scope>
    </KeyValueMapOperations>
     
    

    You can see I am storing the extracted value to a variable that begins with “private.” This is necessary to satisfy the validation for the GenerateJWT policy.

  5. Added a Generate-JWT policy like this

    <GenerateJWT name="Generate-JWT-1">
        <Algorithm>RS256</Algorithm>
        <PrivateKey>
            <Value ref="private.rsakey"/>
        </PrivateKey>
        <Issuer>dinoch-trial-171023@appspot.gserviceaccount.com</Issuer>
        <Audience>https://accounts.google.com/o/oauth2/token</Audience>
        <ExpiresIn>300s</ExpiresIn>
        <AdditionalClaims>
            <Claim name="scope" type="string">https://www.googleapis.com/auth/cloud-platform [https://www.googleapis.com/auth/datastore</Claim>](https://www.googleapis.com/auth/datastore</Claim>);
        </AdditionalClaims>
        <OutputVariable>output_jwt</OutputVariable>
    </GenerateJWT>
     
    

And that worked. I was able to get an RSA-signed JWT with that policy.

The next thing I tried: rather than manually replacing the \n with newlines, and storing ONLY the private key into the encrypted KVM, I did this:

  1. Stored the entire, unmodified credentials.json file into the encrypted KVM

  2. Extracted that into a context variable with a KVM Get like this:

    <KeyValueMapOperations name="KVM-Get-2" mapIdentifier="secrets">
        <ExclusiveCache>false</ExclusiveCache>
        <ExpiryTimeInSecs>300</ExpiryTimeInSecs>
        <Get assignTo="private.credentialsjson">
            <Key>
                <Parameter>dinoch-trial-171023-bdb91206c515.json</Parameter>
            </Key>
        </Get>
        <Scope>environment</Scope>
    </KeyValueMapOperations>
     
    
  3. Ripped the .json into context variables with a JavaScript policy like this:

    <Javascript async="false" continueOnError="false" enabled="true" timeLimit="200" name="JavaScript-1">
        <DisplayName>JavaScript-1</DisplayName>
        <Properties/>
        <Source>
            var c = context.getVariable('private.credentialsjson');
            c = JSON.parse(c);
            for (var prop in c) { 
              context.setVariable('private.' + prop, c[prop]);
            }
        </Source>
    </Javascript>
     
    
  4. Then generated a JWT with a GenerateJWT policy like this:

    <GenerateJWT name="Generate-JWT-2">
        <Algorithm>RS256</Algorithm>
        <PrivateKey>
            <Value ref="private.private_key"/>
        </PrivateKey>
        <Issuer ref="private.client_email"/>
        <Audience ref="private.token_uri"/>
        <ExpiresIn>300s</ExpiresIn>
        <AdditionalClaims>
            <Claim name="scope" type="string">https://www.googleapis.com/auth/cloud-platform [https://www.googleapis.com/auth/datastore</Claim>](https://www.googleapis.com/auth/datastore</Claim>);
        </AdditionalClaims>
        <OutputVariable>output_jwt</OutputVariable>
    </GenerateJWT>
    
    

Both ways worked for me. I suspect that it’s not working for you because of the \n vs newlines, or some other problem transcribing the private key.

Thanks @Dino-at-Google, it was definitely the newline characters and we had just gotten it working. I had also gotten sidetracked as I was going down the path of trying to use GCPs JWT-based OAuth authentication.

Hello @dchiesa1 i’m using Apigee X and wondering if you can elaborate more on step 3 → added THAT ^^ to the Encrypted KVM via the Admin UI, stored under the client_id. how do i do this in Apigee X? the UI only lets you create a key but not the value.

Yes

In Apigee X (or hybrid) there is currently no administrative API for populating a KVM entry. I think the team is working on building out an API to fill this gap. In the meantime, there is a workaround. Look here → devrel kvmadmin reference.

And also, there is a better way to do this in Apigee X. Apigee X now has a possibility to send outbound requests on behalf of a particular Google Cloud Service account, without requiring the API Proxy to obtain a JWT via this grant type. This is described here in the Apigee documentation.

The basic policy configuration you need to use this feature is like this:

<ServiceCallout name='SC-1'>
  <Request>
    <Set>
      <Headers> ... </Headers>
      <FormParams>... </FormParams>
      <Verb>POST</Verb>
    </Set>
  </Request>
  <Response>tokenResponse</Response>
  <HTTPTargetConnection>
    <!-- tell Apigee to invoke this with a Google Access Token -->
    <Authentication>
      <GoogleAccessToken>
        <Scopes>
          <Scope>SCOPE</Scope>
        </Scopes>
      </GoogleAccessToken>
    </Authentication>
    ...
    <Properties>
      <Property name='success.codes'>2xx, 3xx</Property>
    </Properties>
    <URL>https://www.my-site.com/service</URL>
  </HTTPTargetConnection>
</ServiceCallout>

And this requires that you deploy the API Proxy with a service account identity . The documentation I cited above describes the details.

Effectively Apigee will get the access token for you, and send it to the target. So you don’t need to “manually” create and cache your own token in the API Proxy itself. Of course you may still wish to do that (as I described in the steps above). but it’s not required, In Apigee X.

Sorry, I don’t have a screencast / walkthrough of this scenario yet.

1 Like

Thanks so much @dchiesa1 however i’m getting this error

{"fault":{"faultstring":"Google token generation has failed. Please check the authentication configuration.","detail":{"errorcode":"messaging.adaptors.http.filter.GoogleTokenGenerationFailure"}}}

also checked the debug and i can see this error

Failed to generate OAuth2 access token for service account “my SA” scopes [SCOPE] and lifetime 3,600 seconds

@dchiesa1 here is my Service Callout not sure what is going wrong but now getting a 404

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> Service Callout-1 false application/json tokenResponse [https://www.googleapis.com/auth/cloud-platform](https://www.googleapis.com/auth/cloud-platform) 2xx, 3xx [https://us-central1-orgname.cloudfunctions.net/hello-test-apigee](https://us-central1-senso-apigee.cloudfunctions.net/hello-test-apigee)

right, ok. SCOPE needs to be replaced with the appropriate scope for your purpose. For example if you are calling out to GCP Logigng, then the scope ought to be [https://www.googleapis.com/auth/logging.write](https://www.googleapis.com/auth/logging.write).

If you want to use ServiceCallout to connect to a pubsub system , then the scope ought to be [https://www.googleapis.com/auth/pubsub](https://www.googleapis.com/auth/pubsub). If you want to access Google cloud storage buckets, then [https://www.googleapis.com/auth/devstorage.full_control](https://www.googleapis.com/auth/devstorage.full_control).

If you are unsure and you want to use a super broad scope, you can use [https://www.googleapis.com/auth/cloud-platform](https://www.googleapis.com/auth/cloud-platform).


One other thing, aside from token scope. Your Service account ought to be a real service account. Maybe you know this, but just to make sure. If you are seeing “my SA” it’s probably wrong.

Check your Trace. Is it possible the 404 is occurring on the target invocation? Does your proxy have a target? (The serviceCallout is not a target).

If THAT is not the problem, then double check the URL in your ServiceCallout.

BTW I am unsure if the https://www.googleapis.com/auth/cloud-platform scope alone, is suitable to grant authorization for a service account to invoke your own custom Cloud Function. That scope is good for invoking google-provided APIs, like pubsub, storage, Cloud KMS, and so on. If you have your own service, then you may need two things:

  • the token should have cloud-platform scope
  • the service account you are using to invoke the service may need to be assigned a role that contains the cloudfunctions.functions.invoke permission. (details here).

But if it is an authorization problem I would expect 403, not 404.

Also I don’t see you setting the Verb in that ServiceCallout. Also you’re missing the Set element to wrap Headers and Verb (and Path and Payload, if those are applicable).

Check my example, be careful about the XML element hierarchy. Also check the documentation.

@dchiesa1 I tried the service account method, it has limitations in a sense that it does require the service account created from the gcp project where apigee is… in my use case i want to be able to use a cloud function and cloud run from a separate project. I think having KVM with a key from a SA generated from a different project would be powerful - i haven’t really found a nice way to for apigee work with serverless on gcp. can you paste the actually code you used to store the certificate in KVM on apigee… this step is giving me issues – see my command below:

curl -X POST \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $TOKEN" \
    -d '{ "key": "client_id", "value": "-----BEGIN PRIVATE KEY-----" }' \
    "https://$API_HOSTNAME/kvm-admin/v1/organizations/$APIGEE_ORG/environments/$APIGEE_ENV/keyvaluemaps/$KVM_NAME/entries"

it has limitations in a sense that it does require the service account created from the gcp project where apigee is… in my use case i want to be able to use a cloud function and cloud run from a separate project

Ahh, I see. I understand.

I think having KVM with a key from a SA generated from a different project would be powerful - i haven’t really found a nice way to for apigee work with serverless on gcp.

Good feedback. I agree with you.

can you paste the actually code you used to store the certificate in KVM on apigee… this step is giving me issues

Yes. I suggest that you do something a little different. Rather than loading just the private key into the KVM, load the entire JSON. The entire file you’ve downloaded from the cloud console, or created with the gcloud command.

In that case, you want something like this:

curl -i -X POST \
  -H "Authorization: Bearer $TOKEN" \
  $endpoint/kvm-admin/v1/organizations/ORG/environments/ENV/keyvaluemaps/KVMNAME/entries \
  -d key=sakeyjson --data-urlencode value@./my-sacreds.json

That curl command posts a form to the endpoint, telling it what key/value pair to load. The key (name) in this case is sakeyjson. The value of that KVM entry will be the entire contents of the file “my-sacreds.json” in the current directory. That entire contents will be something like this:

{ 
  "type": "service_account",
  "project_id": "cluster",
  "private_key_id": "93158289b2734d823aaeba3b1e4a48a15aaac",
  "client_email": "sa-id-123@ourcluster.iam.gserviceaccount.com",
  "client_id": "31167058558367844",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/sa-id-123%40ourcluster.iam.gserviceaccount.com",
  "private_key": "-----BEGIN PRIVATE KEY-----\nMIIEvgIBADANBgkqhkiG9w0BAQE...8K5WjX\n-----END PRIVATE KEY-----\n"
}

If the KVMNAME in that curl command was “settings”, THEN, at runtime in the API Proxy you need to load that thing into a variable. Like so:

<KeyValueMapOperations name='KVM-Get-SAKey-JSON'>
  <Scope>environment</Scope>
  <ExpiryTimeInSecs>240</ExpiryTimeInSecs>
  <MapName>settings</MapName>
  <Get assignTo='private.sakeyjson'>
    <Key>
      <Parameter>sakeyjson</Parameter>
    </Key>
  </Get>
</KeyValueMapOperations>

But that’s not enough. You then need to extract the things out of that JSON into individual variables. You can do that like so:

<Javascript name='JS-Shred-SAKey-JSON' timeLimit='200' >
  <Properties>
    <!-- this prefix should be "private" to obscure the data in Trace -->
    <Property name='output-prefix'>private</Property>
    <Property name='source'>private.sakeyjson</Property>
  </Properties>
  <Source>
function varname(propertyName) {
  return properties['output-prefix'] + '.' + propertyName;
}
try {
  var obj = JSON.parse(context.getVariable(properties.source));
  for (var p in obj) {
    context.setVariable(varname(p), obj[p]);
    // for diagnostics purposes only. Remove for production use.
    context.setVariable('SHREDDED.' + varname(p), obj[p]);
  }
}
catch (e) {
  context.setVariable('extract_error', "bad inbound message");
  context.setVariable('extract_exception', e.toString());
}
  </Source>
</Javascript>

At this point you have a variable named “private.private_key” holding the PEM-encoded representation of the private key, and other “private” variables holding the other properties from that json. So you can generate the JWT:

<GenerateJWT name="Generate-JWT-2">
    <Algorithm>RS256</Algorithm>
    <PrivateKey>
        <Value ref="private.private_key"/>
    </PrivateKey>
    <Issuer ref="private.client_email"/>
    <Audience ref="private.token_uri"/>
    <ExpiresIn>300s</ExpiresIn>
    <AdditionalClaims>
        <Claim name="scope" type="string">https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/datastore</Claim>
    </AdditionalClaims>
    <OutputVariable>output_jwt</OutputVariable>
</GenerateJWT>
1 Like

@dchiesa1 This is awesome and very well detailed / explained → really really appreciate that you have saved my weeks of pain :slightly_smiling_face: Thank you so so much!

quick question for you: For the javascript file
do i add it as a policy in apigee UI?

1 Like

There is a possibility to add a JS policy via the UI, yes.

The JS policy I showed includes the JS source within the policy config.

<Javascript name='JS-Shred-SAKey-JSON' timeLimit='200' >
  <Properties>
    <Property name='output-prefix'>private</Property>
    <Property name='source'>private.sakeyjson</Property>
  </Properties>
  <Source>
...js source goes here...
  </Source>
</Javascript>

If you like you can also configure a JS policy so that it refers to an “external resource” containing the JS code. To do that, omit the Source element and use the ResourceURL element.

<Javascript name='JS-Shred-SAKey-JSON' timeLimit='200' >
  <Properties>
    <Property name='output-prefix'>private</Property>
    <Property name='source'>private.sakeyjson</Property>
  </Properties>
  <ResourceURL>jsc://name-of-js-resource.js</ResourceURL>
</Javascript>

In this case, the external resource (name-of-js-resource.js) needs to contain your JS source.

@dchiesa1 Thanks! i should also mention that the command above also requires $AUTH header - i have provided below. trying this out right now.

curl -i -X POST \
    -H "Authorization: Bearer $TOKEN" \
"https://$API_HOSTNAME/kvm-admin/v1/organizations/$APIGEE_ORG/environments/$APIGEE_ENV/keyvaluemaps/$KVM_NAME/entries" \
  -d key=sakeyjson --data-urlencode value@./sa.json

oops. yes. Need auth header.

@dchiesa1 thank you so much :clap: i was able to generate a jwt - quick question for you; I would like to then use it to make an api call and pass it as a bearer token - do you know how i can make that happen inside apigee?

worked Flawlessly :clap:

I’m glad to help. re: your “do you know how I can make that happen?” I figure our messages might have crossed, and you might now have figured this out. But just in case, here’s how you can do it with a ServiceCallout.

<ServiceCallout continueOnError='true' name='SC-1'>
  <Request variable='outboundRequest'>
    <Set>
      <Headers>
        <Header name='Authorization'>Bearer {token-generated-from-prior-policy}</Header>
      </Headers>
      <Payload contentType='application/json'>{
    "field1":"something here",
    "foo":"whatever"
}</Payload>
         <Verb>POST</Verb>
         <Path>/hello-test-apigee</Path>
      </Set>
  </Request>
  <Response>apiResponse</Response>
  <HTTPTargetConnection>
    <SSLInfo>
        <Enabled>true</Enabled>
        <IgnoreValidationErrors>true</IgnoreValidationErrors>
    </SSLInfo>
    <Properties>
      <Property name='success.codes'>2xx, 4xx, 5xx</Property>
    </Properties>
    <URL>https://us-central1-orgname.cloudfunctions.net</URL>
  </HTTPTargetConnection>
</ServiceCallout>

The key element here is

<Header name='Authorization'>Bearer {token-generated-from-prior-policy}</Header>

And that token-generated-from-prior-policy is the name of the variable that holds the JWT, that you generated in a prior step. The curly braces tell Apigee to inject that JWT at runtime into the named header. This is because the Header text value is interpreted as a “message template”.

BTW, the Payload element is also treated as a message template.