PBS / Torque Job Submission

The intention of this article is to provide you with an overview into job submission into the PBS / Torque framework.

Please start with loading the Torque environment module so you can access the Torque commands.

module add shared torque

You can submit a job using qsub with the following syntax:
qsub [options] scriptfile

It is mandatory to use a scriptfile since you cannot submit binary files directly to Torque/PBS. The basic structure of the script is shown below:
#!/bin/bash
#$PBS Script options # Optional PBS directives
shellscripts         # Optional shell commands
application          # Application itself

PBS Options

As seen above, options ('directives') can be specified in the Torque/PBS script using "#$PBS". Please note the difference in the jobscript file:

DirectiveShell interpretationPBS interpretation
#CommentComment
#PBSCommentDirective
# PBSCommentComment

The following table shows a summary of some commonly used directives in Torque/PBS. For further information, please refer to the Torque manual.

DirectiveExampleDescription
-N#PBS -N testjobjobname to be used in Torque/PBS
-M#PBS -M user@domainsends email notification to user@domain
-m#PBS -m email is to be sent at the end of the job
-e#PBS -e error-fileredirects error to error-file
-o#PBS -o ~/outredirects output file to $HOME/out
-q#PBS -q all.qspecify queue, i.e. all.q
-d#PBS -d /home/testuserset the working directory
-l#PBS -l walltime=00:30:00resource request, i.e. 30 minutes run time
-l#PBS -l nodes=2:ppn=2request two CPUs on two nodes each

Submitting jobs

You can now submit your jobscript to Torque/PBS using the "qsub" command. Although directives have been specified in the scriptfile, you may add further options such as a specific queue on the command line. The queues available on the cluster can be displayed by executing 'qstat -q':

qstat -q
 Queue            Memory CPU Time Walltime Node  Run Que Lm  State
---------------- ------ -------- -------- ----  --- --- --  -----
default            --      --       --      --    0   0 --   E R
special            --      --       --      --    0   0 --   E R
                                               ----- -----
                                                   0     0

Then submit your job, i.e. directly to the queue "special". Upon successful submission, Torque/PBS responds the job-ID assigned to your job.

qsub -q special testjob.sh
95.master

In order to display the status of your running job(s), use the command 'qstat'. Note, while 'qstat' shows all jobs currently submitted to the cluster, it is convenient to specify the job-ID of the job or to restrict the output to your username (i.e. 'testuser'):

qstat 95
Job id                    Name             User            Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
95.master                 testjob.sh       testuser               0 R special       
 
qstat -u testuser
testuser: 
                                                                         Req'd  Req'd   Elap
Job ID               Username Queue    Jobname          SessID NDS   TSK Memory Time  S Time
-------------------- -------- -------- ---------------- ------ ----- --- ------ ----- - -----
93.master             testuse batch    test2.sh          16452     1  --    --  01:00 C 01:41
95.master             testuse special  testjob.sh        17564     1  --    --  01:00 C 00:01

Job output

During execution (either successful or not), output will be put in your current directory, appended with the job number which is assigned by Torque/PBS. By default, there is an error and an output file for every job.

  • jobname.e#{JOBID}
  • jobname.o#{JOBID}

The output file most likely contains your applications output, whereas the error file can be useful for analysing jobs that failed.

Deleting Jobs

Jobs, either running or being queued can easily be deleted using qdel <job-ID>. Note, you can only delete your own jobs.

qdel 95

Copyright 2002-2014 ClusterVision BV