Grid Engine Job Submission

This article provides an introduction into submitting a job using the Sun Grid Engine framework. If you are interested into running workload through the workload manager this article may help you get things started.

Please start with loading the Sun Grid Engine environment module so you can access the SGE commands.

module add shared sge

You can submit a job with qsub. The qsub command has the following syntax:

qsub [ options ] [ scriptfile | -- [ script args ]]

In general, you will need a jobscript to submit jobs to SGE. A jobscript can contain various settings and variables to go with your application. Basically it looks like:

#$ Script options  # Optional script directives
shellscripts       # Optional shell commands
application        # Application itself

Please see the code examples overview for some commonly used SGE jobscript templates.


Similar to PBS, you can specify options ('directives') with SGE with ”#$” in your script. Please note the difference in the jobscript file:

DirectiveShell intepretationSGE intepretation
# $CommentComment

The following table shows a summary of some commonly used directives in SGE. For further information, please refer to the manual pages of SGE ('man qsub').

-N#$ -N testjobjobname to be used in SGE
-M#$ -M user@domainsends email notification to user@domain
-m#$ -m bemail is to be sent at begin of the job and at the end of the job. use only b for only at beginning of the job or only e for only at the ending of a job
-e#$ -e /tmpredirects error file to /tmp
-o#$ -o ~/outredirects output file to $HOME/out
-q#$ -q all.qspecify queue, i.e. all.q
-cwd#$ -cwd /home/testuserexecute from the current working directory
-pe#$ -pe openmpi 6request a parallel environment, i.e. 'openmpi' using 6 slots (CPUs)
-l#$ -l h_rt=00:30:00resource request, i.e. 30 minutes run time

Note: Resource requests using '#$ -l' strongly depend on your local SGE setup. Please contact your administrator for details about resources that can be requested.

Submitting to a specific queue

Some clusters have specific queues for jobs which run are configured to house a certain type of job: long and short duration jobs, high resource jobs, or a queue for a specific type of node.

To see which queues are available on your cluster use 'qstat':

qstat -g c
CLUSTER QUEUE                   CQLOAD   USED    RES  AVAIL  TOTAL aoACDS  cdsuE  
all.q                             0.94   2048      0    496   2736      0    192 
bigq                              0.96    480      0   1536   2112      0     96 
extraq                            0.91      0      0    432    432      0      0 
smallq                            0.98      0      0    480    480      0      0 
testq                             1.00      0      0    240    240      0      0

Then submit the job, e.g. to the long.q queue:

qsub -q long.q 100

Job output

After completion (either successful or not), output will be put in your current directory, appended with the job number which is assigned by SGE. By default, there is an error and an output file for every job.

  • myapp.e#{JOBID}
  • myapp.o#{JOBID}

Deleting jobs

Removing your job from the system is easy.

qdel <jobid>

Where the job-id is the number assigned by SGE when you submit the job using qsub. You can only delete your own jobs. Please note that deletion takes place regardless of its state, running or spooled.

Copyright 2002-2014 ClusterVision BV