Imagine the following scenario: you are working on an api proxy with your team of developers. All development happens on a single Apigee environment, e.g. “dev”.
- You pick up a new task and start development in a feature branch called feature/1
- Another developer, Joe, picks up the next task and creates a new feature branch called feature/2.
- You hammer your editor locally and put together a partial implementation. You deploy your changes to Apigee.
- Joe is also ready to make his first deployment to test his part of the implementation, so he also initiates his deployment.
- Unaware of what Joe is doing, you fire up your browser/postman/test, etc to see your changes in action. But first you get “ApplicationNotFound” error. You check the output of your deployment script - no errors. You go back to your tests and now you are getting some responses from your proxy but it appears that your changes are not there! You initiate your deployment one more time wondering why it didn’t work the first time.
- Meanwhile Joe is checking his changes and all looks good. He has found a bug and now wants to trace it. He creates a trace session using Apigee Enterprise UI and fires up a couple of requests. But this time, his changes are lost! He can see requests poping up in trace but those are someone else’s requests. He understands that these requests are relevant to the feature you are working on.
- He comes to your desk and hits you in the face (#escalated #quickly)
I can summarise 3 things from this scenario:
- Joe is a bad person - don’t be like Joe.
- You guys use feature branches - which is cool.
- You should configure your deployment scripts so that:
- Each developer can deploy and test their code independent of others.
- CI can deploy and test code independent of developers in all environments.
Solution
What I wanted is to design a deployment pipeline such that whenever a developer deploys a change in a feature branch, that proxy gets deployed with some identifier in the proxy name to separate it from all other deployments. Once feature changes are pushed to a remote branch (for merge/pull requests), CI could deploy its own instance of the proxy again separated from all other deployments. As changes are promoted from one downstream branch to another, CI could then deploy the proxy to relevant environments in an automated fashion.
So to illustrate, here is a table of environments and proxy convention for a sample proxy called “myproxy-v1”:
| Branch | Apigee Environment | Proxy Name | Deployed by |
|---|---|---|---|
| feature/1 | DEV | myproxy-{myname}v1 | me during development |
| master | DEV | myproxy-v1 | CI |
| intg | INTG | myproxy-v1 | CI |
| … | … | … | CI |
| prod | PROD | myproxy-v1 | CI |
This ensures that my deployment and tests do not clash with others’ as it will deploy different api proxies for each team member with separete revisions, code, etc. This method will also ensure only CI can deploy the ‘official’ api proxy with the correct name to an environment.
However just changing the name of the proxy will not be enough. Apigee will block multiple api proxy deployments to the same virtualhost and basepath as Apigee uses virtualhost and basepath pair to uniquely identify the entry point for an api proxy during runtime.
It is not practical to create a new virtual host for each developer, so the deployment script would also need to modify the basepath of the proxy during deployment. This will change the full URL of the resources within that proxy which will fail integration tests hardcoded to use the original URL. So deployment script will also need to modify the request URL used by integration tests.
Implementation
The following implementation is specific to maven but it can be adapted to your tool of choice very easily as solution is nothing but string replacement during deployment.
It is important to understand for below sections that Apigee Maven Plugin copies the proxy files to a temporary folder called “target” and deploys that folder out to Apigee. So string replacements are executed in the target folder without any modification to your original proxy files managed in source control.
Changing proxy name
Maven reads the proxy name to be deployed from //Project/name node in pom.xml. We need to make the value of that element variable depending on who is deploying the proxy. We also need that variable to have a sensible default so that we don’t need to type our name every time we do a feature deployment. Here is how we can achieve it:
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
...
<name>myproxy-${deployment.suffix}v1</name>
...
<properties>
...
<deployment.suffix>${user.name}</deployment.suffix>
</properties>
I am creating a new property called “deployment.suffix” which will have a default value of ${user.name}. By defining this as a property I still have the opportunity to override the value for CI deployments. user.name is a java system property that contains the name of the current user account - see Java System Properties.
So if I deploy the api proxy using the following command, I will get ‘myproxy-oseymenv1’ as the name of the new proxy:
mvn install -Pdev
And if I deploy the api proxy using the following command, I will get ‘myproxy-v1’ as the name of the proxy:
mvn install -Pdev -Ddeployment.suffix=
Don’t worry too much about the “v1” I put as a suffix to the proxy name - it is just a naming convention I follow to keep api proxies in sync with target api versions. You can still use this technique - which is simply inserting a string somewhere in the proxy name - whatever your proxy naming convention is.
Changing the basepath
Basepath of the proxy is configured in default.xml file under apiproxy/proxies folder:
<HTTPProxyConnection>
<BasePath>/customers/v1</BasePath>
<VirtualHost>secure</VirtualHost>
</HTTPProxyConnection>
We want to suffix “/customers/v1” similar to what we did for proxy name. I use Maven Replacer Plugin in order to achieve this.
<plugin>
<groupId>com.google.code.maven-replacer-plugin</groupId>
<artifactId>replacer</artifactId>
<version>1.5.2</version>
<executions>
<execution>
<phase>process-resources</phase>
<goals>
<goal>replace</goal>
</goals>
</execution>
</executions>
<configuration>
<basedir>${target.root.dir}</basedir>
<includes>
<include>apiproxy/proxies/default.xml</include>
</includes>
<replacements>
<replacement>
<token>/customers/v1</token>
<value>/customers/${deployment.suffix}v1</value>
</replacement>
</replacements>
</configuration>
</plugin>
When I deploy the proxy with these settings, I will end up with a proxy called “myproxy-oseymenv1” with the basepath “/customers/oseymenv1”. Joe can now also deploy his changes accordingly and end up with “myproxy-joev1” with basepath “/customers/joev1” - two independent proxies, no fighting. Jenkins, using -Ddeployment.suffix= flag, will create “myproxy-v1” with basepath “/customers/v1”.
But our integration tests are failing - let’s fix that.
Integration tests
I advocate BDD as a development methodology so I build feature files in gherkin syntax to document/discuss requirements with business. These feature files then form the basis of my tests. I use an open source node.js framework called apickli to run the tests which is simple wrapper over cucumber-js with pre-defined gherkin statements to test REST APIs.
I need to be able to change the basepath of the requests initiated by my tests. I don’t want to change any “code” so first thing is to make URL configurable.
Please note that it is again not important which test framework you use - as long as you can make the URL configurable that you can modify with a simple string replacement before deployment. For example, if you are using JMeter, you can move URL to a “User Defined Variable” and replace the value using the method below before deployment.
We don’t want our deployment script to modify files in the original directory so we need to instruct maven to copy the test directory to target folder as well.
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>2.6</version>
<executions>
<execution>
<id>copy-resources</id>
<phase>process-resources</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<overwrite>true</overwrite>
<resources>
<resource>
<!-- source -->
<directory>${project.root.dir}</directory>
<filtering>true</filtering>
<includes>
<include>apiproxy/**</include>
<include>test/integration/**</include>
</includes>
</resource>
</resources>
<!-- destination -->
<outputDirectory>${target.root.dir}</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
As I am using a node.js based tool, I can create a json file to store test configuration:
{
"myproxy": {
"domain": "demo-test.apigee.net",
"basepath": "/customers/v1"
}
}
I can now replace “/customers/v1” to “/customers/oseymenv1” using Maven Replacer Plugin:
... inside maven-replacer-plugin configuration
<includes>
...
<include>test/integration/test-config.json</include>
</includes>
...
<replacement>
<token>/customers/v1</token>
<value>/customers/${deployment.suffix}v1</value>
</replacement>
...
So when tests under target directory are executed by maven, they will make requests to the correct URL for your proxy.