pegasus-exitcode(1)
===================
:doctype: manpage


Name
----
pegasus-exitcode - Checks the stdout/stderr files of a workflow job for any
indication that an error occurred in the job. This script is intended to be 
invoked automatically by DAGMan as the POST script of a job.


Synopsis
--------
[verse]
*pegasus-exitcode* [*-h*][*-t* 'n'][*-r* 'rv'][*-n*] 'job.out'


Description
-----------
*pegasus-exitcode* is a utility that examines the STDOUT of a job to 
determine if the job failed, and renames the STDOUT and STDERR files 
of a job to preserve them in case the job is retried.

Pegasus uses *pegasus-exitcode* as the DAGMan postscript for all jobs 
submitted via Globus GRAM. This tool exists as a workaround to a 
known problem with Globus where the exitcodes of GRAM jobs are not 
returned. This is a problem because Pegasus uses the exitcode of a 
job to determine if the job failed or not.

In order to get around the exitcode problem, Pegasus wraps all 
GRAM jobs with Kickstart, which records the exitcode of the job 
in an XML invocation record, which it writes to the job's STDOUT. 
The STDOUT is transferred from the execution host back to the submit 
host when the job terminates. After the job terminates, DAGMan runs 
the job's postscript, which Pegasus sets to be *pegasus-exitcode*. 
*pegasus-exitcode* looks at the invocation record generated by 
kickstart to see if the job succeeded or failed. If the invocation 
record indicates a failure, then *pegasus-exitcode* returns a non-zero 
result, which indicates to DAGMan that the job has failed. If the 
invocation record indicates that the job succeeded, then *pegasus-exitcode* 
returns 0, which tells DAGMan that the job succeeded.

*pegasus-exitcode* performs several checks to determine whether a job 
failed or not. These checks include:

. Is STDOUT empty? If it is empty, then the job failed.
. Are there any `<status>` tags with a non-zero value? If there are, 
then the job failed. Note that, if this is a clustered job, there could 
be multiple `<status>` tags, one for each task. If any of them are 
non-zero, then the job failed.
. Is there at least one `<status>` tag with a zero value? There must 
be at least one successful invocation or the job has failed.

In addition, *pegasus-exitcode* allows the caller to specify the exitcode 
returned by Condor using the *--return* argument. This can be passed to 
*pegasus-exitcode* in a DAGMan post script by using the `$RETURN` variable. 
If this value is non-zero, then *pegasus-exitcode* returns a non-zero result 
before performing any other checks. For GRAM jobs, the value of `$RETURN` will 
always be 0 regardless of whether the job failed or not.

Also, *pegasus-exitcode* allows the caller to specify the number of successful 
tasks it should see using the *--tasks* argument. If *pegasus-exitcode* does not 
see N successful tasks, where N is set by *--tasks*, then it will return a 
non-zero result. The default value is 1. This can be used to detect failures 
in clustered jobs where, for any number of reasons, invocation records do 
not get generated for all the tasks in the clustered job.

In addition to checking the success/failure of a job, *pegasus-exitcode* 
also renames the STDOUT and STDERR files of the job so that if the job is 
retried, the STDOUT and STDERR of the previous run are not lost. It does 
this by appending a sequence number to the end of the files. For example, 
if the STDOUT file is called "job.out", then the first time the job is run 
*pegasus-exitcode* will rename the file "job.out.000". If the job is run again, 
then *pegasus-exitcode* sees that "job.out.000" already exists and renames 
the file "job.out.001". It will continue to rename the file by incrementing 
the sequence number every time the job is executed.


Options
-------

*-h*::
*--help*::
Prints a usage summary with all the available command-line options.

*-t* 'n'::
*--tasks* 'n'::
Number of tasks expected. If less than 'n' tasks succeeded, then 
*pegasus-exitcode* will fail with a non-zero return value. This is used in
cases where we may not get a Kickstart invocation record for some tasks.
Normally Seqexec will detect failed Kickstart invocations and fail
accordingly.

*-r* 'rv'::
*--return* 'rv'::
Return value reported by DAGMan. This can be specified in the DAG 
using the $RETURN variable. If this is non-zero, then *pegasus-exitcode*
immediately fails with a non-zero return value itself.

*-n*::
*--no-rename*::
Don't rename 'job.out' and 'job.err' to '.out.XXX' and '.err.XXX'. 
This option is used primarily for testing.


Authors
-------
Gideon Juve `<juve@usc.edu>`

Pegasus Team <http://pegasus.isi.edu>

