KServe provider now exposes LLM autoscaling with Workload Variant Autoscaler and HPA/KEDA

The KServe provider now supports automatic scaling for LLM inference workloads through Workload Variant Autoscaler (WVA), Horizontal Pod Autoscaler (HPA), and KEDA triggers.

Previously, LLM deployments ran at a fixed replica count regardless of actual traffic, leading to either under-provisioned latency spikes or over-provisioned resource waste.

You can now configure autoscaling rules directly in the KServe provider spec. To learn more, see the release notes.