Paxata
Case Study
Paxata, a leading data preparation platform for data analytics, was handed over to Devopsbay for maintenance, testing, automation, and cost-cutting purposes. The project aimed to enhance the platform's efficiency through automation, ensure seamless upgrades, and reduce operational costs.
Key aspects of the project
Platform Maintenance and Automation
Implementing CI pipelines for preparing and deploying new releases.
Automating Deployment Processes
Ensuring smooth and automated deployment workflows.
Tests Management
Writing new tests and removing irrelevant ones, and handling zero downtime Kubernetes upgrades.
Cost Optimization
Downscaling tenants' instances that were not being used.
Technical challenges
- Lack of Autoscaling: Addressing the absence of auto scaling features.
- Zero Downtime Upgrades: Ensuring zero downtime while scrolling over clients' instances after upgrades.
- Cluster Creation: Building new clusters from scratch without any manual steps.
Project implementation stages
- 1
Setting Up CI/CD
Establishing continuous integration and continuous deployment pipelines for new releases.
- 2
Automation for Cluster Creation
Automating the creation of new clusters, initially for testing Kubernetes upgrades.
- 3
Kubernetes Upgrade Preparation
Preparing dependencies and procedures for Kubernetes upgrades.
- 4
Scale-Down Procedure
Creating and automating the scale-down-to-0 procedure for unused instances.
Problems and solutions
Manual Operations and Tribal Knowledge
The infrastructure, initially provisioned in an Infrastructure as Code (IaC) manner, had accumulated many manual operations over the years, making cluster creation time-consuming and error-prone. Devopsbay automated these steps, reducing errors and speeding up the process.
CI/CD Process Issues
The CI/CD process was plagued with false negatives and positives, making releases difficult. Devopsbay overhauled the CI/CD pipelines, reducing manual hotfixes and streamlining release management.
Hidden Issues in Source Code
Fixed image versions and container repository names in the source code caused issues, such as hitting Docker Hub limits during node rollovers. Devopsbay addressed these hidden issues, ensuring smoother deployments.
Team involvement
- Project Manager
- Lead Software Developer
- Lead DevOps Engineer
- QA Engineer
- Software Developers
- DevOps Engineer
Technologies Used
Kubernetes
AWS
Kustomize
Jenkins
GitHub
Rust
Helm
Calico
NGINX
Bash Scripting
Results and feedback
Devopsbay successfully added new features to the Paxata platform, created robust CI/CD pipelines for releases and cluster creation, established a process for Kubernetes upgrades, and automated the hibernation of unused tenant instances. These improvements led to enhanced platform efficiency, reduced operational costs, and a more streamlined deployment process.
Conclusion
The Paxata project showcased Devopsbay's expertise in automating and streamlining complex software development processes. By addressing key technical challenges and implementing effective solutions, Devopsbay significantly improved the platform's performance and cost-efficiency, ensuring a seamless experience for Paxata's users.
