Questão de System Administration #019e3a4f
Let's assume that an organization needs to process large amounts of data in bursts, on a cloud-based Kubernetes cluster. For instance: each Monday morning, they need to run a batch of 1000 compute jobs of 1 hour each, and these jobs must be completed by Monday night. What's going to be the most cost-effective method?
Run a group of nodes with the exact required size to complete the batch on time, and use a combination of taints, tolerations, and nodeSelectors to reserve these nodes to the batch jobs.
Leverage the Kubernetes Cluster Autoscaler to automatically start and stop nodes as they're needed.
Commit to a specific level of spending to get discounted prices (with e.g. “reserved instances” or similar mechanisms).
Use PriorityСlasses so that the weekly batch job gets priority over other workloads running on the cluster, and can be completed on time.
Explicação Oficial / Explicações da Comunidade
The most cost-effective method for handling burst workloads on a cloud-based Kubernetes cluster is to use the Kubernetes Cluster Autoscaler to automatically start and stop nodes as needed. This approach dynamically adjusts the cluster size based on pending pods, ensuring that resources are provisioned only when required and scaled down during idle periods, minimizing costs.
How it Works
The Cluster Autoscaler monitors pod scheduling status. When pods cannot be scheduled due to insufficient resources, it triggers the cloud provider's API to add new nodes. Conversely, when nodes are underutilized for a configurable period, it removes them. This aligns resource consumption with actual demand, eliminating waste from over-provisioning.
Why the Correct Answer is Right
- Cost Efficiency: The Cluster Autoscaler ensures that nodes exist only when needed. For the burst scenario, nodes will be added Monday morning to handle the 1000 jobs and removed after completion, avoiding charges for idle resources.
- Automation: No manual intervention is required; the autoscaler reacts to workload changes automatically.
- Cloud-Native Integration: It works with major cloud providers (AWS, GCP, Azure) and respects instance types, zones, and scaling policies.
Why the Incorrect Answers are Wrong
- Running a group of nodes with exact required size using taints, tolerations, and nodeSelectors: This approach requires manual sizing and provisioning. If the workload varies or the estimate is off, resources may be wasted or insufficient. It also lacks dynamic scaling, leading to potential over-provisioning.
- Committing to a specific level of spending with reserved instances: Reserved instances offer discounts for steady-state workloads but are inflexible. For burst workloads, you pay for capacity that is idle most of the week, making it cost-ineffective.
- Using PriorityClasses to give batch jobs priority: PriorityClasses affect scheduling order but do not add or remove nodes. If the cluster lacks capacity, high-priority pods may still remain pending. This does not address resource provisioning and can lead to incomplete jobs or preemption of other workloads without cost savings.