Among cloud customers, we see that almost 90% have setup 1 or 2 Orgs and 95% have 3 Orgs or less. We also see that almost 80% of customers have between 1-3 Envs per Org.
Please share some reasons for how and why you’ve setup your Orgs/Envs the way you have.
Customers would like:
To keep production traffic not being impacted by non-production traffic.
To separate prod MP IP’s from non-production MP IP’s for IP whitelisting.
There might be other reasons, but these are important.
Here’s what I have seen and what makes sense in terms of isolation and data segregation, there might be other reasons which are program specific or customers’ business specific:
Why multiple orgs:
- You have multiple API programs that are managed by different groups
- You have multiple API teams or multiple vendors working in an API program
- You do not want to share data (apps, keys, developers, apiproducts etc.)
- You do not want to share org level resources like KVM, resourcefiles etc.
- You may or may not want to share apiproxies (not a big deal, since you can control visibility with RBAC and deployments using env)
- Some customers would buy a performance pack to run performance tests
- You may have different business units within the same company, so you want different orgs for each of the business units.
Why multiple envs:
- You want to share kms data such as apps, developers, keys, apiproducts etc.
- You want to share org level resources such as KVM, resource files - javascript, javacallouts etc
- You do not want to share Cache resources, so you define cache resource for each env
- You want to deploy different version of your proxies to different environments depending on what phase you are in (dev, test, , int, perf etc)
- You want to track analytics for your prod and non-prod environments separately
One key thing here is consolidation. If you aim to consolidate the resources spread across multiple orgs into fewer orgs (one or more) it would be virtually impossible. Whereas with multiple environments you would get the isolation you need (similar to versioning your resources, caches, proxies etc.). Also if you are a cloud customer with so many orgs then you eventually end up sharing the same routers and MPs across different orgs which starts hurting you because you are no longer physically isolated.
Here’s the setup we commonly advocate with customers as a starting point.
This is assuming a default Edge licence in the cloud which comes with 2 Orgs and 6 Environments. We’re assuming the two orgs are used for administrative separation between non prod and prod.
| Environment | Apigee Target | Test Data | Type of Testing | Apigee Org |
|---|---|---|---|---|
| DEV | Mock | Data and behaviour from Mocks | Data and behaviour | Non Prod |
| TEST | TEST | Data and behaviour from TEST | Partial integration | Non Prod |
| PERF | TEST / PERF | Data and behaviour from TEST / PERF | Performance | Non Prod |
| UAT | UAT | Data and behaviour from UAT | Smoke | Non Prod |
| PRE | PROD | Data and behaviour from PROD | Smoke | Prod |
| PROD | PROD | Data and behaviour from PROD | Smoke | Prod |
Here’s some background on why we advocate this approach.
The above assumes that testing in DEV in against target mocks where we have full control over the behaviour and data being exposed. This allows full testing of both the data expected to be mediated by the API and behaviour of the API in happy path, error and edge case scenarios (e.g. back end is down). Test can be fully automated with the confidence that the mock target data and behaviour will not change unexpectedly.
In TEST the APIs are pointing to a real TEST Target. Typically we’ll have less control over the behaviour and data available and so the number of tests that can be automated be be limited compared to DEV where we have full target control. As many tests should be configured here as possible give the reliability of the test system.
In PERF performance tests will be run. Typically a small subset will be automated as part of the promotion of the APIs towards production but there may also be a larger set of tests that cannot be automated that can be run on a more ad hoc basis as necessary e.g. a change has been made that is likely to have a significant performance impact. The idea here is that gross changes in API performance are caught by the automated tests and the full performance tests are only run as required.
In UAT, PRE and PROD a smaller subset of tests are run that are testing basic API setup and functionality without covering all scenarios.
The PRE environment is a mirror of prod and is used for final testing before any launch.
In some scenarios the above assumptions may not hold true in which case the environment for a certain kind of testing may change but the principals should remain the same.
The above setup also lends itself to automation as there’s a clear path to production.
Whilst a full Continuous Delivery may be difficult for some organisations due to constraints outside the API Program this setup does allow you to build pipelines for as much of the journey as possible with maybe only the final step requiring some manual intervention to trigger the final production push.
Out of the box the traffic between the two orgs is sharing a common infrastructure but the above setup also allows a simple separation should traffic isolation become a priority. Typically we see the environments in the prod org and non prod org separated to isolate production traffic.
See http://apigee.com/about/specification-sheets → Edge Cloud Traffic Isolation Pack for more details on traffic isolation.
Would love to hear back from folks who have >1 Org.
What would you miss if you only had 1 Org?
@Dom Couldwell, @Vishal Agrawal
I would miss having the separation of concerns with the out-of-the-box roles. To support non-prod and prod in a single organization, there would have to be corresponding roles for each with appropriate restrictions. I think this could be accomplished by adding custom non-production roles based on the existing roles. That doesn’t seem that hard to manage on the surface, however I’ve not done this in practice, yet.
Great discussion! I have a question regarding “internal” consumers vs “external” consumers given the solution above. We have a client where some APIs must be consumed by employees (internally) only. If you follow the pattern above, then what is the recommendation to separate the KVMs, Caches, client ids, etc. from internal consumers vs external consumers?
I know that you could prefix KVM and Cache names with internal/external, but they are effectively shared because they are in the same Cassandra ring in production. There could be a case where you don’t want the Internal/External resources to be located on the same ring. Are there any other recommendations, other than purchasing an additional organization pack or isolation pack?
Thanks Dom - this is very clear. When you use an environment strategy like this, is it possible to insert a new revision into just certain - more mature environments? Like - if I need to make an emergency hotfix that applies to only pre-prod and prod, how does that work?
Short answer is yes. We have to cater for the reality where we need to hotfix one environment.
Long answer is how you achieve this with a suitable Continuous Integration strategy including Source Control Management (SCM), automated deployment and testing.
This will be the subject of some more in depth articles and when they get published I will try to cross post here but the crux is being able to have a clear path to production for the ‘happy path’ SDLC but also cater for the hot fix scenario. Key in the end is that the two approaches are complementary i.e. if you apply a hotfix how to you make sure it gets reapplied to your main development stream. This is not an API specific issue, all software development must address this so pick a SCM strategy and stick with it e.g. GitLab Flow or Git Flow
If the question is relating to traffic isolation then yes a traffic isolation pack is the correct approach. If the question is more related to administrative separation then this is more of a governance issue and would need to be addressed differently. See the community thread discussing One API Team or Many for more ideas on managing governance.
In this setup how would you handle the “try it” feature of interactive docs, or in other words how would you suppor sandbox functionality?
That’s a great question. Time has moved on in terms of Apigee licencing and now we have more flexibility in terms of the numbers of Orgs and Environments that Customers can use. What I see most commonly is that we add one more environment to the prod org called sandbox which is used for exactly the purpose that you mention.
Nice, that’s the conclusion I came to also.
Good descriptive information! one question on above, what is the best practice to expose org for external app users on dev portal? Could you please provide more details on it? Thank you!