Traditionally, when a service provider launches a service, it is not uncommon to see data from different customers stored together in the same database or storage instance. Therefore it is normal that the service account application maintains full access to the database or storage instances. This centralized design aggregates the burden of maintaining security for the service, given application logic is the only way of defense against unauthorized access to data.
In this article, we would like to introduce an architecture which compartmentalizes customer data and processing workloads at infrastructure level, so that the application and data storage is separate from other customers. This capability provides additional assurance on data separation, and improves the security by reducing the attack surface to sensitive customer data, while maintaining the operation efficiency (similar to the centralized design) for the service provider.
As a Software as a Service (SaaS) provider (‘provider’) who builds their service on GCP, it will be good to have a service-oriented architecture which is modularized and scalable to support a seamless customer onboarding experience. In this document, we are going to share an architecture to address it.
Architecture
Based on the above customer requirements, we would like to share the high level conceptual architecture view of the design. To better illustrate the idea, this design only layouts the general separation design and deliberately ignores other specific application design details.
Diagram 1 - Overall Architecture
As we depicted in the diagram above, there are 3 objectives that constitute the overall architecture design:
- Segregation of Data Storage (Boxes on the right) - we would recommend to leverage GCP projects to segregate customer data storages, as separation is by default ensured within GCP for resources between projects.
- Dedicated Access Control (Gray box in the middle) - we would recommend to use dedicated service account(s) for each customer and manage it centrally to ensure consistency in IAM operations.
- Runtime Environment Separation for Applications (boxes on the left) - One of the key objectives is to build a separation mechanism to isolate application workload for different customers. Depending on the choice of services for application runtime, there are different ways to implement application separation.
This design creates the ‘compartment’ for each SaaS user and ensures user data is logically separated among each user. For details about each objective and how it can be implemented in Google Cloud, Please refer to the following sections.
Segregation of data storage
Data storage will be separate from the application runtime environment. Ideally the application should be stateless to support the scalability requirement of the customer requirements.
Dedicated data project(s) will be created for each customer, which hosts all the private customer data uploaded / processed by the service. This includes all the security services (such as Cloud IAM, Cloud KMS, etc) associated with any data services in scope.
Access to data service instances should be managed by Cloud IAM service of each data project, and only allow service accounts dedicated to the corresponding application for access. In other words. The IAM binding between application tier and data tier will be dedicated per customer, or more securely, per workload.
It is highly likely that the provider will run its own set of services to support some common needs among their customers. To address the need, dedicated data project(s) should be created for shared services data, and access to data should be restricted to corresponding applications as well.
Dedicated access control
| Note: Choice of authentication and authorization mechanism for the application tier is considered out of scope of this architecture. In principle, this architecture should be able to work with any application access control mechanism the service provider decides to implement. The Access Control model we are discussing here focused on service to service authentication between Application tier and Data Storage. |
|---|
As the application workloads and data storage is logically separated among customer tenants, it makes no sense to share any access control policies, therefore, any service accounts created on Google Cloud for customer specific data storage should be solely used by the designated application workload of the same customer. Service account accessing across customer tenants must be strictly prohibited.
In case a customer’s private workload needs access to a shared service provided by the Service Provider, it should be explicitly approved and follow the standard access pattern defined by the Service Provider, e.g.: through an application API or explicit approval for accessing a shared database.
For best practices regarding using service accounts on GCP, please refer to this guide.
Runtime environment separation for applications
We are expecting the application workloads to be atomic and modularized which can be logically segregated among different customers. Depending on the different choice of technology stack, each customer’s application workload will be run separately in their own logically isolated ‘compartments’.
In this section, let’s zoom in to the application segregation design, and focus on how to create these ‘compartments’ among different types of workloads. Two common scenarios covers:
For workloads running on a multi-tenant GKE cluster
For containerized workloads, the following diagram shows an architecture for a multi-tenant GKE cluster that uses GKE namespaces to segregate workloads among customers:
Diagram 2 - GKE Workload Segregation
For each customer, a customer specific namespace will be created and customer selected workloads modules will be deployed into this namespace. Service providers can create namespace(s) for platform-wide shared services as well.
We strongly recommend using Workload Identity to access data services stored in data projects. For each customer namespace, a dedicated set of service accounts should be created, and service accounts must not be shared by workload from different namespaces.
For VM/GCE based workload
For VM/GCE based workloads, the following diagram shows an architecture with Shared VPC:
Diagram 3 - VM/GCE Workload Segregation
In this scenario, a shared VPC will be created in the host project and will be centrally managed by the SRE team. Applications will be deployed to VM instances hosted in a service project based on customer selected service modules. VM instances must not be shared among different customers. Similar to the namespace for service providers, service providers can also use their own service project to run shared services for all the customers.
The network administrators will use secure tags for firewall policies to regulate traffic between customer and service provider’s workloads.
As customer workloads will be running on VMs, they will be provisioned as part of a customer specific service project and nested in the assigned subnets, as determined by the Share VPC configuration.
Service accounts will be created and attached to the VM, based on the application deployed on VM, within the service project and it will be bound with IAM policy to enable accessing data services in the customer data project respectively.
Use case
For the ease of understanding, we would like to describe the problems statement as:
As a Software as a Service (SaaS) Provider on GCP, I would like to have a flexible and scalable application architecture which can address customer-specific security and compliance requirements on their data without impacting other customers.
To address the above problem statement, we would like to highlight the design requirement as below:
- Logical separation among customers
- Centralized security control by the Service Provider
- Least Privilege and Modular deployment of service
- Streamlined customer lifecycle management
For the ease of elaboration, the following personas are referenced:
- Service Provider: the SaaS provider that builds service offerings on GCP and offers the service to their Customers;
- Customer: the end user (a corporate or enterprise customer) of the SaaS offered by the Service Provider
- Cloud Service Provider (CSP): The cloud platform provider supporting the SaaS provider, under the scope of this document, this refers to GCP.
Design considerations
Security, privacy, and compliance
As this architecture ensures logical separation among customers’ data and the application workloads, it provides unique benefits from security, privacy and compliance perspective.
The service provider will be able to offer a wide range of ‘on-Demand’ security controls to their end customers, this includes implementing independent or even conflicting requirements from different customers, such as data locality, cryptographic key management, data retention / lifecycle management, and access control requirements.
For certain capabilities, such as Customer Managed Encryption Key (CMEK), the architecture allows service providers to directly relay Google Cloud APIs to the end customers, without rebuilding the whole encryption layer from scratch.
To expand visibility and control to the end customer, the Service Provider can also benefit from this architecture to build their own platform transparency services, by leveraging Google Cloud Data Access Logs (for Service Provider’s access to customer data) and Google Cloud Access Transparency and Access Approval (for Cloud Provider Access to data).
Cost optimization
Note: This architecture itself does not deal with cost optimization directly, and we are expecting cost considerations to be addressed by the Service Provider in the detailed application solution design.
It is worth mentioning that by implementing logical separation among different customers, the service provider will be able to easily monitor the Google Cloud cost associated with each individual customer and react accordingly.
As a general guidance on how to monitor and optimize cost on Google Cloud, Please refer to Cost Management.
Operational efficiency
As this architecture compartmentalizes customers’ data and the application workloads, it streamlines the Customer Lifecycle Management process for the Service Provider.
To onboard a new customer will include the following steps:
- Initial private compartments for both applications and data storage;
- Provision resources in each segment in respect of customer needs;
- Setup the IAM binding for application and storage in the private segment;
- Setup IAM binding for shared services.
When a customer decides to leave, the service provider can provision the resources used by the customer and prove to the customer that their data has been safely removed without impacting any existing customers.
This can be fully automated with an application and infrastructure CI/CD Pipeline, which can be extended to cover other customer lifecycle milestones as well.
Deployment
Note: This architecture is a high-level conceptual design which is not deployable directly, and we are expecting deployment automation considerations to be addressed by the Service Provider in the detailed application solution design.
To deploy this architecture, complete the following tasks:
- Complete the detailed component view and infrastructure design for the overall solution architecture.
- Determine the choice of CI/CD tooling and deployment pipeline(s) for the infrastructure and application.
- Plan, design, and implement the deployment automation solution for the overall application solution.
Most Google Cloud Services support deployment automation using Terraform. For more information, see Terraform on Google Cloud.
What’s next
Learn more about the Google Cloud products used in this architecture:
For reference architectures, design guides, and best practices, explore the Cloud Architecture Center.