Common Infra
Case Study
Common Infra was a project aimed at creating a multi tenant SaaS platform for all DataRobot applications. This involved consolidating services into a shared Kubernetes cluster and database infrastructure on AWS. Devopsbay was tasked with designing, implementing, and maintaining this platform to ensure efficient resource management and seamless operations.
Key aspects of the project
Autoscaling
Configuring and managing Karpenter for autoscaling, considering the diverse requirements of different services.
Infrastructure Migration
Transitioning from disparate infrastructure to a common multitenant environment, ensuring proper separations and adherence to Kubernetes best practices.
Platform Design and CI/CD
Designing and implementing the common platform along with CI/CD pipelines for multiple environments and regions.
Developer Support
Supporting developer teams responsible for each service and maintaining the overall platform.
Technical challenges
- Resource Allocation: Ensuring services have adequate resources to start and run efficiently.
- Workload Calibration: Calibrating resource requests and limits for workloads.
- CI/CD Process: Building a CI/CD process that targeted only the affected environments.
- Service Migration: Assisting teams in properly configuring services post-migration to Kubernetes.
Project implementation stages
- 1
Tool Selection
Choosing tools for policy management, autoscaling, and resource provisioning.
- 2
CI/CD Pipeline Development
Building pipelines for cluster creation, service deployment, and third-party resource management based on Git changes.
- 3
Service Migration
Migrating services from old platforms to the new common infrastructure.
- 4
Cluster Creation
Establishing common infra clusters for new regions.
- 5
Platform Maintenance and Development
Ongoing maintenance and development of the platform.
Problems and solutions
IaC Structure
Creating an efficient and idempotent Infrastructure as Code (IaC) structure for multiple environments.
Resource Calibration
Fine-tuning services for optimal resource consumption.
Resource Creation
Establishing resources from the VPC level in other regions and accounts.
Multi-Cloud Support
Extending the common platform to other cloud providers like Azure and GCP.
Team involvement
- Multiple developer teams
- DevOps Engineers
- DevOps Manager
Technologies Used
Kubernetes
AWS
Helm
GitOps (Flux)
Kyverno
Karpenter
GCP
Azure
NGINX
Results and feedback
Devopsbay successfully created CI/CD pipelines for provisioning common infra Kubernetes clusters across multiple environments and regions (US, EU, and Japan). The infrastructure allowed for proper autoscaling and efficient node placement for services. Additionally, Devopsbay developed processes for creating clusters for single-tenant SaaS on other providers like Azure and GCP, ensuring a scalable and flexible infrastructure.
Conclusion
The Common Infra project demonstrated Devopsbay's capability in designing and implementing a robust multitenant SaaS platform. By addressing key technical challenges and providing comprehensive support, Devopsbay ensures a seamless transition to a common infrastructure, enhancing resource management and operational efficiency for DataRobot applications.
