{"id":"scheduler","level":"datacenter","name":"Cluster scheduler and resource allocation","shortName":"Scheduler","role":"Matches queued workloads to available compute resources under operational policies.","description":"A cluster scheduler turns shared hardware into an organized service. Jobs request resources such as CPUs, GPUs, memory, and time, and the scheduler allocates a compatible set when policies and availability permit. Slurm, one common example, represents GPUs as schedulable resources and exposes assigned devices to job steps. The atlas shows requests entering a queue, resource groups being reserved, and workers launching on those allocations. It distinguishes this job-level decision from the GPU hardware scheduler that issues instructions and from the serving scheduler that batches inference requests. Good placement can improve communication locality, but a scheduler cannot remove the algorithm's communication requirements or manufacture capacity that the cluster does not have.","specs":[{"label":"Example system","value":"Slurm GRES scheduling","context":"GPU-aware resource allocation."},{"label":"Allocation inputs","value":"Resource request + policy + availability","context":"Actual rules depend on the installation."},{"label":"Boundary","value":"Job placement","context":"Distinct from kernel instruction scheduling."}],"interfaces":[{"name":"Job requests and cluster state","direction":"in","signal":"Resource requirements and availability"},{"name":"Worker allocation","direction":"out","signal":"Assigned execution resources"}],"tradeoff":"Packing improves utilization. Queue fairness, locality, and job constraints can compete.","whyAI":"Large training jobs need coordinated resource reservations. Allocation decisions influence waiting time and communication placement.","related":["rack","parallelism","storage","checkpoint","serving"],"sources":[{"title":"SchedMD — Slurm Generic Resource Scheduling","url":"https://slurm.schedmd.com/gres.html"},{"title":"MIT — Queueing Models and Little’s Law","url":"https://web.mit.edu/1.041/www/lectures/L8-queuing-models-2024sp.pdf"}],"scale":"Cluster-wide logical control plane","kind":"scheduler","mechanism":["Read a job’s resource request and policy constraints.","Choose an available compatible allocation.","Launch and track the job while releasing resources when it finishes."],"misconception":"Cluster scheduling and GPU instruction scheduling occur at different levels of the machinery.","check":{"question":"What does a cluster scheduler primarily decide?","options":["Which available resources a job receives","The position of every electron in a GPU","The next token probability inside a model"],"answer":0,"explanation":"It allocates and manages execution resources; lower software and hardware layers execute the assigned workload."},"branch":"System operations","subbranch":"Workload allocation","science":[{"title":"Little’s law","equation":"L = λ W","explanation":"Average in-system job count L equals admitted arrival rate λ times average time W spent within the chosen system boundary. It links workload concurrency to waiting and service time.","assumptions":"Stable long-run averages with consistent arrival and population definitions; not a prediction of each job’s latency or an overloaded transient queue."}],"history":[],"learningObjective":"Use Little’s law with a declared queue boundary.","engineeringExample":"At 100 requests/s and 200 ms average latency, average in-flight work is about 20 requests.","evidenceNotes":"Stationarity and a well-defined system boundary are required.","visualFamily":"model-workload","prerequisites":["host-cpu"],"labId":"library","scope":"No current calculator directly models this lesson’s main mechanism. Use the library for related bounds while retaining their stated omissions.","investigationId":"execution","labScope":"No current calculator directly models this lesson’s main mechanism. Use the library for related bounds while retaining their stated omissions."}