Versions Compared

    Key

    • This line was added.
    • This line was removed.
    • Formatting was changed.
    Comment: Published by Scroll Versions from this space and version 1.0

    The next step is to build a more robust dependency system. system 

    Building on what we have learnt in the previous tutorials this tutorial will: 

    • Give example of JobLabel for linking jobs 
    • Give example of Dependency linking to JobLabel
    • Shows correct dependency graph in the Qube! GUI

    Feel free to download and an run the script below. It sets up below script as it setup a job that will :

    • Create a Parent "Sleep job" with a range of 60 
    • Create a Blocked Child "Sleep job" that links to the Parent job and waits for a complete status before starting

    Advanced_DependencyDependancy.py


    Code Block
    languagepython
    #!/usr/bin/python
    
    
    # Below are required imports for the script to run
    import os, sys
    
    
    # The below few lines of code are to determine the OS of the machine that your running 
    # this script from and then define the location of the Qube! API
    if 'QBDIR' in os.environ:
    	sys.path.append('%s/api/python' % os.environ['QBDIR']);
    elif os.uname()[0] == 'Darwin':
    	sys.path.append('/Applications/pfx/qube/api/python');
    elif os.uname()[0] == 'Linux':
    	sys.path.append('/usr/local/pfx/qube/api/python');
    else:
    	sys.path.append('c:/program files/pfx/qube/api/python');
    
    
    # The below line of code is to import the above defined Qube! API
    import qb
    
    
    # Below is the main function to run in this script 
    def main():
        
        # ----------------Start creation of Parent Job----------------------------------------
        
        # Below defines an empty list for combining all tasks in the dependancy chain
        task = []
        
        # Below creates an empty dictionary to be filled by the following lines of code 
        job = {}
        
        # Below defines a label for the dependancy to be used internally within this script
        job['label']= 'ParentLabel'
        
        # Below defines the name of the Qube! job 
        job['name'] = 'python parent job'
        
        # Below defines how many Instances/subjobs the job is to spawn
        job['cpus'] = 1
        
        # Below defines the internal Qube! jobtype to be used to execute the job
        job['prototype'] = 'cmdrange'
        
        # The below parameters are explained further in the "Job submission with job package explained" page
        package = {}
        job['package'] = package
        job['package']['cmdline'] = 'sleep QB_FRAME_NUMBER'
    	
    	# Below defines the Agenda/Range of the job this will fill the Frames/Work section of the Qube! GUI
    	# "0-60x10" is range 0-60 in chunks of 10 frames
        agendaRange = '0-60x10'
        
        # Below defines the internal command required to generate the agenda 
        agenda = qb.genframes(agendaRange)
        
        # Below defines the job details for the agenda 
        job['agenda'] = agenda
        
        # Below appends the details of this task to the job dictionary for later submission
        task.append(job)
    	
    	# ----------------Start creation of Child Job----------------------------------------
    	
    	
    	# Below creates an empty dictionary to be filled by the following lines of code
        job = {}
        
        # Below defines a label for the dependancy to be used internally within this script
        job['label']= 'ChildLabel'
        
        # Below defines the dependancy of this job see below for possible dependancy strings
        job['dependency'] = 'link-complete-job-ParentLabel'
        
        # Below defines the name of the Qube! job
        job['name'] = 'python child job'
        
        # Below defines how many Instances/subjobs the job is to spawn
        job['cpus'] = 1
        
        # Below defines how many Instances/subjobs the job is to spawn
        job['prototype'] = 'cmdrange'
        
        # The below parameters are explained further in the "Job submission with job package explained" page
        package = {}
        job['package'] = package
        job['package']['cmdline'] = 'sleep 20'
        
        # Below appends the details of this task to the job dictionary for later submission
        task.append(job)
    	
    	# Below submits the task list to Qube! 
        listOfSubmittedJobs = qb.submit(task)
        
        # Below prints out a list of jobs that have been submitted by name 
        for job in listOfSubmittedJobs:
        	print '%(name)15s: %(id)s' % job
    
    # Below runs the "main" function 
    if __name__ == "__main__":
        main()
        sys.exit(0)
    
    

    This script differs from the rest quite quiet a lot:lot 


    task = []

    To create a list of jobs that are submitted:

    task.append(job)

    Combines the list of jobs for final submission with:by 

    listOfSubmittedJobs = qb.submit(task)

    With this method the jobs are not submitted per "job = {}" instead combined and submitted once all tasks have completed:completed 

     

    job['label']= 'ParentLabel'

    This creates a internal label for the job which is assessed at submission time:

    job['dependency'] = 'link-complete-job-ParentLabel'

    ...

    This will run once the Parent job is complete.

    job['dependency'] = 'link-failed-job-ParentLabel'

    This will run once the Parent job is failed.

    job['dependency'] = 'link-killed-job-ParentLabel'

    This will run if the Parent job has been killed.

    job['dependency'] = 'link-done-job-ParentLabel'

    This will run once the Parent job returns a status of done. Done means if the job completes,fails or has been killed.

     

    You can also link jobs by different types.types 

    job['dependency'] = 'link-done-work-ParentLabel'

    This will run depending on the status of the jobs work.work 

    job['dependency'] = 'link-done-subjob-ParentLabel'

    This will run depending on the status of the jobs subjobs.

    See Also

    Job Dependency Attribute Syntax 

    Continue to