KEDA is a Kubernetes-based Event Driven Autoscaler. It works alongside standard Kubernetes components like the Horizontal Pod Autoscaler.
The Autoscaling Service in Ansys HPC Platform Services supports scaling of Kubernetes Deployments, Stateful Sets, and Jobs using KEDA. The autoscaler scales based on the Prometheus metric. It pushes the number of pending jobs metric periodically (every 10s) to Prometheus. For each project that has jobs pending for evaluation, the number of pending jobs metric is pushed to Prometheus.
One KEDA deployment file is created and applied on a Kubernetes cluster using the kubectl command for each project's task definition. The number of pending jobs metric will be pushed with additional attributes like project ID, task definition ID, and applications specified in the task definition.
KEDA will periodically query the Prometheus server for the value of the metric and will either scale up or down evaluator instances based on the metric value.
When there are no pending jobs for evaluation for a project's task definition, the autoscaler will delete its KEDA deployment file applied on the cluster.
Autoscaling Kubernetes Jobs
When evaluators need to be spawned to execute the pending jobs of a specific project and task definition, use the scaling Jobs mode.
When querying Prometheus server for the number of pending jobs metric value, KEDA will pass the project ID and task definition ID in the query.
The number of maximum evaluator instances that can be spawned per project's task definition is configured via the autoscaling configuration.
Autoscaling Kubernetes Deployments and Stateful Sets
When evaluators need to be spawned to execute the pending jobs of all projects that specify the same application in the task definition, use the scaling Deployments and StatefulSets mode.
Kubernetes Deployments and StatefulSets must be deployed on the Kubernetes cluster namespace where they will run.
When querying the Prometheus server for the number of pending jobs metric value, KEDA will pass the application name in the query.
The number of maximum evaluator instances that can be spawned per Deployment or StatefulSet is configured via the autoscaling configuration.