Icon

This is the documentation for an older version of Qube. The latest version of the documentation can be found here: Qube

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3

The next step is to understand basic dependancies 

Feel free to download an run the below script as it will only setup a job that will  run a "sleep" command on the worker 

Basic_Dependancy.py

 

below dependencies 

In this example, we use the job's dependency field to set up a job-based dependency.  Job based dependencies are those that will wait for the entire dependent job to finish before the current job starts.  There are also subjob (instance) and agenda (frame) based dependencies.

Below is the code with commented explanations of its contents contents

Code Block
titlejobSubmit_dependency-example.py
languagepython
#!/usr/bin/python

import os, sys

# BelowThe arenext requiredfew importslines forattempt to import the scriptQube toAPI. runIf importthe os,path systo the #qb Themodule
below# fewis linesnot ofin code$PATH areor to$PYTHONPATH, determinewe thewill OSattempt ofto thefind machineit thatby yourlooking runningin known
# thislocations
scripttry:
from and then define theimport locationqb
ofexcept theImportError:
Qube! API if 'QBDIR' inif os.environ:
	sys.get("QBDIR"):
        qbdir_api = os.path.append('%s/api/python' % join(os.environ['QBDIR']);
elif os.uname()[0] == 'Darwin':
	sys.path.append('.get("QBDIR"),"api","python")
    for api_path in (qbdir_api,
                     "/Applications/pfx/qube/api/python');
elif os.uname()[0] == 'Linux':
	sys.path.append('/",
                     "/usr/local/pfx/qube/api/python');
else:
	sys.path.append('c:/program files/pfx/qube/api/python');/",
     # The below line of code is to import the above defined Qube! API import qb "C:\\Program 
# Below is the main function to run in this script 
def main():Files\\pfx\\qube\\api\\python",
                     "C:\\Program  # ----------------Start creation of Parent Job----------------------------------------Files (x86)\\pfx\\qube\\api\\python"):
        if api_path not in sys.path and os.path.exists(api_path):
    # Below creates an empty dictionary to be filled by the following lines of code sys.path.insert(0,api_path)
           job =try:
{}          # Below defines the name of theimport Qube!qb
job      job['name'] = 'python parent job'  except:
       # Below defines how many Instances/subjobs the job is tocontinue
spawn     job['cpus'] = 1     break
    # Belowthis definesshould thethrow internalan Qube!exception jobtypeif towe've beexhuasted usedall toother executepossibilities
the job   import qb
job['prototype'] = 'cmdrange'
def main():
    
    # TheWe're belowgoing parametersto aresubmit explainedtwo furtherjobs in the "Jobexample submission- withthe jobsecond packagewill explained"wait
page    # packagefor =the {}first
 
   job['package'] = package
    job['package']['cmdline'] = 'sleep QB_FRAME_NUMBER'
	
	# Below defines the Agenda/Range of the job this will fill the Frames/Work section of the Qube! GUI
	# "0-60x10" is range 0-60 in chunks of 10 frames
    agendaRange = '0-60x10'
    
    # Below defines the internal command required to generate the agenda 
    agenda = qb.genframes(agendaRange)
    
    # Below defines the job details for the agenda 
    job['agenda'] = agenda
	
	# Below creates an empty list filled by the following lines of code
    listOfJobsToSubmit = []
    
	# Below evaluates the Parent job to be submitted and adds the to the above list
    listOfJobsToSubmit.append(job)
    
    # Below evaluates the Parent job to be submitted and adds the to the above list
    listOfSubmittedJobs = qb.submit(listOfJobsToSubmit)
    
    # Below calls the Parent job to be submitted and then prints the job ID # Set up the basic job properties as we've done before.  We're going to do
    # a "sleep" job, so we'll limit this test to osx or linux
    job = {}
    job['name'] = 'python parent job'
    job['cpus'] = 2
    job['prototype'] = 'cmdrange'
    job['requirements'] = 'host.os=linux or host.os=osx'
    
    # For the package, we'll just do a simple sleep
    package = {}
    package['cmdline'] = 'sleep 5'
    job['package'] = package

    # Create a typical agenda, as we've done in previous examples.
    agendaRange = '0-60x10'
    agenda = qb.genframes(agendaRange)
    job['agenda'] = agenda

    # Now we submit a single job (as a single element list).  
    # This will be the parent job
    listOfJobsToSubmit = []
    listOfJobsToSubmit.append(job)
    listOfSubmittedJobs = qb.submit(listOfJobsToSubmit)
    parentJobID = listOfSubmittedJobs[0]['id']
    print 'parent: %d' % parentJobID
	
	# ----------------Start creation of Child Job----------------------------------------
	
	
	# Below creates an empty dictionary to be filled by the following lines of code
    job = {}
    
    # Below defines the name of the Qube! job#=================================================================#
 
    # Now we'll create a second job that will be dependent on the first
    # the basic setup is the same as before
    job = {}
    job['name'] = 'python child job'
    job['cpus'] = 1
  # Below defines how many Instances/subjobs the job is to spawnjob['prototype'] = 'cmdrange'
     job['cpusrequirements'] = 1
  'host.os=linux or host.os=osx'
 
      # BelowTo definescreate howthe manydependency Instances/subjobson the jobparent, iswe tosimply spawncreate a
   job['prototype'] = # 'cmdrangedependency' attribute.  See link in this doc's text for #more Belowinformation
defines the jobs dependancy which links back to the first job in this script
    job['waitfordependency'] = 'link-done-job-%d' % parentJobID
    
     # TheContinuing belowon, parameterswe arecreate explaineda furthertypical in the "Job submission with "sleep" job packagefor explained"testing
page     package = {}
    jobpackage['packagecmdline'] = package'sleep 20'
    job['package']['cmdline'] = 'sleeppackage
20'    
      # BelowAs createsbefore, anwe emptysubmit listthe filledsingle byjob theas followinga linessingle ofelement codelist
    listOfJobsToSubmit = []
    
    # Below evaluates the Child jobs to be submitted and adds the to the above list
    listOfJobsToSubmit.append(job)
    
    # Below calls the Child jobs to be submitted and then prints the job IDs
    listOfSubmittedJobs = qb.submit(listOfJobsToSubmit)
    
    for job in listOfSubmittedJobs:
        print 'child: %d' % job['id']

# Below runs the "main" function 
if __name__ == "__main__":
    main()
    sys.exit(0)

A non edited version of this script can be found along with others :

  • Windows - C:\Program Files\pfx\qube\examples\jobSubmit03.py
  • OSX - /Application/pfx/qube/examples/jobSubmit03.py
  • Linux - /usr/local/pfx/qube/examples/jobSubmit03.py

Online - Python

Running this script will create 2 jobs.  The first will run immediately, the second will be in a blocked state until the first job completes, then it will automatically start running.

For more information on the dependency attribute syntax, see Specifying Job Dependencies

Continue to Advanced Dependencies