Our GPU nodes typically have 8 GPUs per node. Is it possible to create a reservation for just a subset of those GPUs? I see from the manual that trackable resources ("TRES") can be specified when creating a reservation, but not generic resources ("GRES"). I'd like to do something like: sudo scontrol create reservation=gpu1 starttime=2023-07-20T06:00:00 endtime=2023-07-31T06:00:00 users=foobar partition=gpu Nodes=node10 gres=gpu=1 This fails because "gres" is not accepted for reservation creation, so I must set aside all 8 GPUs: sudo scontrol create reservation=gpu1 starttime=2023-07-20T06:00:00 endtime=2023-07-31T06:00:00 users=foobar partition=gpu Nodes=node10 I can't be the first person that has asked about this? Is there a work-around to reserve just a subset of the GPUs on a node?
Jeff, please see bug#5771comment#2. Quoted comment below. We are actively looking into adding this feature in 23.11. > Currently, Slurm can’t explicitly create reservations for GRESs. Supported resources > that can be reserved include cores, nodes, licenses, burst buffers, and features. > However, you could possibly work around this limitation by creating reservations based > off a feature. To do that, set a feature for nodes that have GPUs with a string like > “k80” and issue a command like the following: > scontrol create reservation starttime=now nodes=all duration=15 features=k80 user=root > Just be aware that “the reservation creation request can [only] identify... *one* > feature that every selected node must contain.” > See https://slurm.schedmd.com/reservations.html. *** This ticket has been marked as a duplicate of ticket 5771 ***
Thank you.
