Summary: | Limit interactive jobs on partition | ||
---|---|---|---|
Product: | Slurm | Reporter: | Daniel P Davis <daniel.p.davis> |
Component: | Configuration | Assignee: | Alejandro Sanchez <alex> |
Status: | RESOLVED INFOGIVEN | QA Contact: | |
Severity: | 4 - Minor Issue | ||
Priority: | --- | CC: | alex, charles.wright |
Version: | 16.05.0 | ||
Hardware: | Linux | ||
OS: | Linux | ||
Site: | EM | Alineos Sites: | --- |
Atos/Eviden Sites: | --- | Confidential Site: | --- |
Coreweave sites: | --- | Cray Sites: | --- |
DS9 clusters: | --- | HPCnow Sites: | --- |
HPE Sites: | --- | IBM Sites: | --- |
NOAA SIte: | --- | OCF Sites: | --- |
Recursion Pharma Sites: | --- | SFW Sites: | --- |
SNIC sites: | --- | Linux Distro: | --- |
Machine Name: | CLE Version: | ||
Version Fixed: | 16.05.0 | Target Release: | --- |
DevPrio: | --- | Emory-Cloud Sites: | --- |
Description
Daniel P Davis
2016-09-16 07:52:18 MDT
Hi Daniel, right now there's no partition/qos limit to accomplish this. We can mark it as a possible future enhancement if needed. Daniel, note that the job_submit plugin has access to the job_desc->script field to discern interactive from non-interactive jobs. You could check that field to control the submissions to specific partitions as desired. Dunno exactly what you want to limit: - "Number of interactive jobs submitted to specific partition, computing also past jobs" - "Number of interactive jobs RUN/PEND to a specific partition at the same time" - "Reject any submission to specific partition if job is interactive" - ...? We are trying to reject interactive job submission to one of our partitions. Can you point me to documentation/examples on how to use the job_submit plugin? Sure, this is the documentation: http://slurm.schedmd.com/job_submit_plugins.html You should configure the JobSubmitPlugins=lua in your slurm.conf. There are a few examples in the contribs/lua directory: https://github.com/SchedMD/slurm/tree/master/contribs/lua I think something in this direction is what you want: --[[ Example Lua script ]]-- function slurm_job_submit( job_desc, part_list, submit_uid ) local partition = "part1" if ((job_desc.script == nil or job_desc.script == '') and job_desc.partition == partition) then slurm.log_info("slurm_job_submit: interactive job submitted by user_id:%d to partition:%s rejected", job_desc.user_id, job_desc.partition) return slurm.FAILURE end return slurm.SUCCESS end function slurm_job_modify( job_desc, job_rec, part_list, modify_uid ) return slurm.SUCCESS end slurm.log_info("initialized") return slurm.SUCCESS This is great info. Only one thing is unclear for me. Is the job_submit plugin build as part of the default installation, or something I need to compile myself? By default Slurm does not load any Job Submit Plugins. To accomplish this task, set JobSubmitPlugins=lua in slurm.conf and reconfigure it. Then Slurm will dynamically execute the lua script named "job_submit.lua" located in the default script directory (typically the subdirectory "etc" of the installation directory). Lua is an interpreted scripting language and needs no compilation after source edition. Thanks. FYI, I just tried the script that comes with 16.05.10 and got this slurmctld[4133]: error: lua: /etc/slurm/job_submit.lua: /etc/slurm/job_submit.lua:5: ')' expected near '!' It appears that != in lua is "~=" http://lua-users.org/wiki/LuaTypesTutorial diff job_submit.lua job_submit.lua-new 5c5 < if ((job_desc.script == nil or job_desc.script == '') and job_desc.partition != partition) then --- > if ((job_desc.script == nil or job_desc.script == '') and job_desc.partition ~= partition) then |