Cloud-native technologies empower organizations to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infrastructure, and declarative APIs exemplify this approach.
These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil.
Applications have become increasingly complex with users demanding more and more. Users expect rapid responsiveness, innovative features, and zero downtime. Performance problems, recurring errors, and the inability to move fast are no longer acceptable. They'll easily move to your competitor.
Cloud native is about speed and agility. Business systems are evolving from enabling business capabilities to being weapons of strategic transformation that accelerate business velocity and growth. It's imperative to get ideas to market immediately.
Here are some companies who have implemented these techniques. Think about the speed, agility, and scalability they've achieved.
Title
Schema Migration of Huge number of databases in Azure
Problem Statement
Our client was struggling with schema migration of their Databases which is hosted in Azure. The number of databases is in millions and due to this huge amount, the schema migration used to take up to a few days Solution: The solution was to design a scalable architecture that can scale out to handle request traffic of size comparable to that of twitter. At the same time, we had to make the system reliable and traceable with status update of each run and proper logging and monitoring on different levels.
Since the actual requests to be processed were in millions, we proposed to incorporate an azure service bus in order to improve the control on queuing and processing (retry if required as an example).
Obviously, we need to provide scaling super power to the system to meet with the massive traffic. We deployed our resources in an Azure managed kubernetes service (AKS) to leverage the scaling.
A RunManagement Api generated the queue in service bus which will further be picked up and processed by an Azure Function App called Worker App. While processing each message, the WorkerApp gets required details (like what script to run on which databases) and runs those migration scripts on the given databases.
We use a couple of other Apis to fetch scripts and also for updating the status of script runs. Sailing on the kubernetes scaling platform, we make use of external metrics autoscaling called KEDA (k8s Event Driven Autoscaling) along with the native k8s HPA (Horizontal Pod Autoscaling) to achieve the project scale out requirements most efficiently.