Creating Submission Dialogs

    Submission UIs are driven by basic Python [ordered] dictionaries.  In this section, we will describe how to create and install submission UIs.

    There exists a working, well documented example called skeleton.py in the distribution's submit_scripts directory.  This directory is most easily found by going to File > Open Submit Scripts directory... 

    Icon

    After your submission script has been created and "installed" to the correct directory, see Command Line Submission for instructions on how to launch and populate your new submission dialog directly from a 3rd Party application, i.e. Maya, AfterEffects, or custom, in-house software.

     


     

     

    Contents

     

     


    Terms used in this document

    • Parameter: A container of fields. A parameter typically defines a label (displayed on the left) and at least one data field, lined up in a row on the right of the parameter container.
    • Field: A field holds data that will be used by the submission dialog to create a job in Qube.
    • Qube tabs are "stock" tabs.  Qube tabs contain all qube-specific parameters/fields, things like job name and worker selection. All Qube tabs and all parameters within them will automatically be a part of every submission dialog.
    • User tabs are created by submission scripts. They will contain parameters specific to this submission dialog (and therefore the application being driven by this dialog). All developer-generated content will exist in the user tabs.

    Script Location

    Submission UIs are loaded by ArtistView at startup, though they can also be invoked from the command line through the qubeSubmission[.exe] executable.  For a submission UI to be visible in either location, it must live in a location about which ArtistView knows. Default locations for submission scripts are:

    WindowsC:\Program Files\Pfx\qube\qubeArtistView\submit_scripts
    OS X/Applications/pfx/qube/qubeArtistView.app/Contents/MacOS/submit_scripts
    Linux/usr/local/pfx/qube/qubeArtistView/submit_scripts

    User-defined submission scripts can be defined in ArtistView preferences, under the "Submission" tab:

    Submission scripts can be easily found by going to File > "Open Submission Script Directory..." in ArtistView.


    Dialog Definition

    The most basic submission dialog is a python dictinoary.  It must be defined in a .py file within one of the submit script directories and assigned to a root-level variable called dialogs. The dictionary must be keyed on some submit-script name with a value that, itself, is a dictionary with at least one key called "parameters" who's value, again, is a dictionary.  For example:

    Multiple dialogs can exist in the same submission script by simply adding more keys at the "my_dialog" level.

    Dialogs can be defined as basic dictionaries, though basic dictionaries have no sense of order, so when order is important, use an OrderedDict.  Depending on the version of Qube you are using, you may need to "from util.av_collections import OrderedDict" at the top of your script to have access to OrderedDicts.  OrderedDicts are accessed exactly like dictionaries, though they are formed more like functions.  

    For example, if this is how one creates a dictionary called 'd':

    ...then this is how one creates an OrderedDict called 'd':

    The primary difference is that when we iterate through the OrderedDict, the keys will maintain order, whereas a basic dict does not guarantee this behavior.

    Dialog Top Level Attributes

    Submission UIs have the following attributes that are defined as keys at the top level (in the example from the last section, the "top level" of the submission UI is the same level as "parameters").

    displayDisplay name - this shows up in menus and title bars. If not provided the script's key name will be used.
    shortNameShort display name - this will show when there is little space (i.e. tab). If not provided, the display name will be used.
    prototypeThe job's prototype, i.e. cmdline, cmdrange, appFinder, pyCmdLine, pyCmdRange, maya, etc
    helpContent for help button. Can be plain text or basic html. The final product will be html wrapped in <pre> tags.
    hasRangeShould we present a frame range widget?  True/False (if omitted, then False)
    canBatchShould we present a range execution widget?  True/False (if omitted, then False)
    canPadFramesShould we present a frame padding widget?  True/False (if omitted, then False)
    rangeRequiredMust the user provide a frame range?  True/False (if omitted, then False)
    hasThreadControl

    Should we preset thread controls?  "all"/"specific"/False (if omitted, then False).

    • "all": presets both "use all threads" and "specific thread count" widgets
    • "specific": presents only the "specific thread count" widget
    • False: provides no threading widgetry
    preDialogFunction to be run before the dialog is displayed
    preShowValidateValidation function to check data sent to the submission dialog. If validation fails, the dialog will not be displayed.
    postDialogFunction to be run immediately after the dialog is displayed
    preSubmitFunction to be run immediately before the job is submitted
    preSubmitValidateValidation function that is run prior to submission - designed to validate submission options. If validation fails, the job will not be submitted.
    postSubmitFunction to be run immediately after the job is submitted (having access to the newly submitted job)
    categoryCategory in the Submit pulldown menu in the main interface
    parametersPlaceholder for our OrderedDict of parameters.
    hiddenShould this submission dialog show up in the main interface's Submit menu?

    Dialog Pre/Post Functions

    A dialog can make use of setup, teardown, and validation functions.  These functions are defined at the top level of the dialog (as mentioned above).  The functions are:

    Function NameArguments / ReturnDescription
    preDialog
    • Arguments:
      • dialog - the current, fully assembled dialog before it is shown. As a developer, you have access to all fields and their values to manipulate them as necessary prior to submission. Access to the passed-in job object (i.e., what would be passed in to the submission dialog on the command line via --submitDict) can be had through dialog.getJob().
    • Returns:
      • Nothing. Return values are ignored.

    This function is designed to do final setup of the dialog prior to display.

    For example, for Maya submissions, the user may want to change the current renderer from the setting that is stored in Maya to one of the other options that are available in the scene. During submission, all possible renderers are encoded into the passed-in job object which preDialog parses and adds each item to a drop-down menu in the submission dialog.

    preShowValidate
    • Arguments:
      • dialog - the current, fully assembled dialog, before and preferences are applied. Access to the passed-in job object (i.e., what would be passed in to the submission dialog on the command line via --submitDict) can be had through dialog.getJob().
    • Returns:
      • True - validate passed. The dialog will be displayed
      • Anything other than True - validation failed. If the return value was a string, display that string in an error dialog, otherwise display a generic error dialog. The error dialog will have the options, "OK" and "Ignore". If the user chooses "OK" or closes the dialog, submission will be cancelled and no dialog will be displayed. If the user chooses "Ignore", then the error will be ignored, and the dialog will be displayed despite the failed validation.

    This function is intended to provide a mechanism to validate settings that may be necessary prior to the dialog being displayed.

    For example, if a scene has not yet been saved, or if a setting within the scene will prevent it from rendering. In those cases, the developer would return some sort of a message from this function which will then be visible to the user who can then solve the problem and try their submission again.

    postDialog
    • Arguments:
      • dialog - the current, fully assembled dialog, in its final state prior to submission.
      • job - a dict of the qb.Job that will be used in submission, but prior to that job containing the rendering command line (if one exists) and before the job's agenda has been created.
    • Returns:
      • Nothing. Return values are ignored.

    PostDialog is run after the user clicks submit, but before the cmdline or agenda has been created. It is designed to allow the developer to manipulate data prior to the creation of the the cmdline and agenda.

    For example, AfterEffects (AE) can render out to either an image sequence or a movie file. An image sequence can be distributed across many computers (each getting a frame or group of frames), whereas a movie file can only be rendered by a single machine (an AE limitation, not a Qube limitation). The AE submission dialog's postDialog determines the output type and changes the instance count to 1 and moves from rendering multiple frames to rendering a single partition (all frames on one agenda item) when the output type is a single movie file.

    preSubmit
    • Arguments:
      • dialog - the current, fully assembled dialog, in its final state prior to submission.
      • job - the fully assembled, ready-to-render dictionary representation of the qb.Job that will be submitted via qb.submit().
    • Returns:
      • Nothing. Return values are ignored.

    This function is designed to allow access to manipulate job data prior to submission. Unlike postDialog, preSubmit is run after the agenda has been created and after the command template has been formed - in other words, It is run after the job-to-be has been fully realized.

    For example, Cinema 4D does not log the path to the file it has written - as such, Qube cannot determine the output path by parsing the outputlog; instead, the output paths are pre-populated into each agenda item prior to submission. This is done by preSubmit.

    preSubmitValidate
    • Arguments
      • dialog - the current, fully assembled dialog, in its final state prior to submission.
      • job - the fully assembled, ready-to-render dictionary representation of the qb.Job that will be submitted via qb.submit().
    • Returns:
      • True - validate passed. The job will be submitted.
      • Anything other than True - validation failed. If the return value was a string, display that string in an error dialog, otherwise display a generic error dialog. The error dialog will have the options, "OK" and "Ignore". If the user chooses "OK" or closes the dialog, submission will be cancelled and user will be brought back to the submission dialog where they can correct the problem and try again. If the user chooses "Ignore", then the error will be ignored, and the job will be submitted, just as if validation passed.

    This function is the final validation step prior to submission. It allows the developer to, e.g., double check any generated values, check the existence of output paths, check permissions, verify the correct options were chosen, etc.

    For example, within the last few AfterEffects (AE) releases, when creating an output from a render queue, the "Save in subfolder" option is turned on by default. This will tell AE to create a subfolder in the output directory that is named after the composition. When rendering through AE's UI, this directory is created if it does not exist, but when rendering via the command line, aerender will NOT create the directory, thus causing the render to fail. Qube works around this by using preSubmitValidate to verify that the path exist prior to submission.

    postSubmit
    • Arguments:
      • dialog - the current, fully assembled dialog, in its final state prior to submission.
      • submitted - the return from qb.submit(). On successful submission, this will be a list of qb.Jobs (which can be cast as dictionaries).
    • Returns:
      • Nothing. Return values are ignored.

    PostSubmit is run after the job has been submitted to and accepted by the supervisor. The passed-in "submitted" argument contains a list of jobs that were accepted by the supervisor. When submitting one job at a time, "submitted" will be a list of one job that is fully formed. When submitting more than one job at a time (in the same submission dialog - which would be rare), then the return will be a placeholder job, with only the job id and submitted time values populated. This function allows the developer to perform any sort of clean-up or notification about the job's existence.

    For example, after a job has been submitted, one might want to update a production tracking system, or send an email, or clean-up local temporary storage, etc.

     

     

    Dialog Parameters

    All parameters are described by python dictionaries, keyed on the parameter name. Each parameter defines several top-level attributes and a dictionary of field names keyed on the field name. For example:

    The above code block would define a parameter with both the internal name and the display name of "simple_path" but contains no fields.

    If "fields" is a top-level attribute of a parameter, then other available top level attributes are:

    fieldsAn [Ordered]dictionary of parameter fields. See next section for more information
    hideIf true, the parameter cannot be seen by the user
    labelThe name displayed in the UI. If not included, the key name of the parameter's dictionary will be used
    requiredIf True, then the parameter label will be red and the job will not submit while any parameter in the field is empty
    tooltipText displayed when the user hovers over the parameter. Fields can optionally provide their own tool tips in addition to that parameter's tool tip.
    tabThe tab on which the parameter will reside. Excluding "tab" will cause the parameter reside in the main lower tab.

     

    Parameter Fields

    Parameters are containers for fields.  Fields hold the actual data that will eventually be submitted to the farm.  Like all parts of the submission UI, fields are python dictionaries, keyed on the field name. For example:

    The above code block will define a "path" field called "simple_field" that has no label.

    If "opttype" is a top level attribute of a field, then other available top level attributes are:

    always_useAlways include this field and its data in the submitted job, even if they are left blank/unchagned by the user.
    argThe command line argument to which this field should map. "Arg" is only necessary if mapping directly to a command line argument.
    defaultThe default value for his field. Note: this is not the same as "value". "Default" means "this is the value the application expects, therefore the application will operate the same if this value is excluded." When using a default value, the value will be displayed in the submission UI, but will only be passed to the job if changed to a non-default value. There is never a need to use both a "default" and a "value".
    destinationWhere this key/value pair should be included in the job being submitted. For example, "Priority" has a destination of "job", as it will live at the top of the job object. "Maya executable" will have a destination of "job.package", as it will live in the job's package. Options that are used only by callback functions may not need to live in the job at all – those would have a destination of None.
    destination_typeCurrently, this can either only be excluded or set to the Python type "dict". If set to dict, then the field value is expected to be a python dictionary. This is only valid for opttype "combo" fields.
    labelIf provided, the field will get its own label to the left of the field. As an example, see "Farm Concurrency" in any submission dialog. "Instances:" and "Max:" are field labels.
    optionsopttype-specific options. See below for more information.
    opttypeThe type of field to use. See below for more information.
    valueThe value that should be displayed in this field when the dialog is launched. Unlike "default" values, a "value" will always be passed to the job, even if not changed. There is never a need to use both a "default" and a "value".
    update_functionCallback function that is fired any time a field in this field's watch_list is updated.
    watch_lista list of fully qualified field names that determine the value of this field. For example, in a "Maya Jobtype" submission dialog, if the user clicks the "Each instance uses all cores" option in the Qube parameters, then the Maya parameter "Threads" must be set to 0 to instruct Maya to use all available cores on the worker. If the user unchecks "Use all cores", then changes the "Threads per instance" value, then the Maya parameter "Threads" must be set to match the "Threads per instances" value. In this case, the Maya "Threads" parameter would have both the "Threads per instance" and "Each instance uses all cores" fields in its watch_list.

     

    Field opttypes and options

    All fields are defined the same way, yet not all fields are the same. For example, there's no reason to define a number range for a string field, and there's no reason to define default text for an integer field. All field opttypes have their own set of options

    opttypedefinition and options
    int

    A number. The presented widget will only accept a number and will provide up and down arrows to change the value

    Options:

    • range: A dictionary who value is another dictionary with range attributes:
      • min: the minimum allowable value (default: 0)
      • max: the maximum allowable value (default: 99)
      • skip_min: the lower range of excluded values
      • skip_max: the upper range of excluded values

    Example:

    Would create:

    • An integer field called "cc_year" with no label
    • destined for the job's package
    • with a start value of the current year
    • that watches a field called "use_cc.fields.use_cc"
    • ...and calls the "updateCCYear" function if the "use_cc.fields.use_cc" field changes
    • and would allow numbers 0-2030, excluding 1-2013.

    string

    A single-line string field. The presented widget will be a simple, single-line text-entry field.

    Options:

    • placeholder: Text that will be displayed while the field is empty and out of focus. For example "Search..."

    Example:

    Would create:

    • A string field called "name" with no label.
    • destined to be a job's top-level attribute.
    • with "Enter Job Name..." as placeholder text.

    text

    Multi-line string field. The presented widget will be a multi-line text-entry field.

    Options:

    • None

    Example:

    Would create:

    • An empty text field called "regex_errors"
    • Destined for the job's package

    combo / combo_path

    A combo box (or drop-down) field. The presented widget will be a typical combo box, without icons, with a down-arrow on the right that, when clicked, will show all available options.

    "combo" and "combo_path" are identical with the exception that the value passed to the job from a combo_path will be treated as a path (being automatically converted by the worker at render time, if requested), whereas a combo's value will be treated as a string.

    Options:

    • items: A list of available items in the pulldown menu. The list can consist of strings or 2-item tuples. In the case of the tuple, the first will be the value displayed in the dropdown box, while the second item will be the value used in the job, e.g [ ("SD", "640x480"), ("HD", "1920x1080") ]. Using the return of a function is acceptable here, e.g. my_populateComboBox_function().
    • editable: If True, the user can type in their own value in the text field in addition to choosing an option from the dropdown list. If False, then only the items in the dropdown box may be used.

    Example:

    Would create:

    • A combo field called "combo_mutable" with no label.
    • destined to be a job's package.
    • that is user-editable, in other words, a user can type into the field as well as select one of the pulldown options
    • where the dropdown will display "Choice 1 display name", "choice2", and "choice 3". If the user chooses "Choice 1 display name", then the job's package will contain the value "choice1".

    table

    An editable table of data. The presented widget will be a table of text-entry fields.

    Options:

    • dimensions: a tuple of (rows,columns) that defines the initial size of the table. default: (2,2)
    • visible_rows: the number of visible rows. If the table is larger than the number of visible rows, a scrollbar will be visible.
    • items: a list of list of items to display in the table, where each row is defined by a list of strings

    Example:

    Would create:

    • An table text field called "linux_env" with no label
    • The data from this table will not be used directly in the job, rather, it will be processed into the job by the submission script's preSubmit function. Therefore, its destination is None.
    • The table will have 20 rows and 2 columns, though only 4 rows will be visible, so a scrollbar will be present.

    datetime

    A calendar field. The presented widget will look like an editable combo field, but will present a calendar when the down-arrow is clicked.

    Options:

    • format: the accepted date format. Default is "yyyy/MM/dd hh:mm"

    Example:

    Would create:

    • A datetime field called "start_time" with no label.
    • The data from this field will not be used directly in the job, rather, it will be processed by the submission script's preSubmit function. Therefore, its destination is None.
    • This field will update itself if the field "delayed_start.fields.delayed_start" is changed. When that field is changed, the updateStartTime function will be called.
    • Acceptable text input must be in the form "yyyy/MM/dd hh:mm"


    Dialog manipulation

    Submission dialogs must be able to be updated programmatically while being used. In addition to setup, for example, it is quite common for rendering options to be mutually exclusive or for one option to effect another.  In those cases, the developer must be able to both monitor specific fields for changes and change fields as necessary.  We will describe the various methods for doing so in this section.

    Parameter and Field Addresses

    In order to make changes, one must be able to find the field and/or parameter of interest. Parameter and field addresses (aka path) follow the form:

    From outside the submission dialog
    <submission_name>.parameters.<parameter_name>.fields.<field_name>[.<attribute>]
    From within the submit script
    <parameter_name>.fields.<field_name>[.<attribute>]

    For example, the "Farm Concurrency" parameter has two fields, labelled with "Instances:" and "Max:".  The parameter is defined as such:

    To gain access to the field labelled "Instances:", we could use the address concurrency.felds.cpus.

    To gain access to the field labelled "Max:", we would use the address concurrency.fields.max_cpus.

     

    Dialog Accessor/Helper Functions

    The dialog object (that is passed into most functions that are run by the submit script) has a few helpful functions for accessing widgets and/or their values.

    archiveJob( job )When passed in a qb.Job, this function will display a "File Save" dialog box, allowing that job to be written to disk as a binary Qube job archive (.qja)
    enableField ( path, enable=True, tooltip=None)Enabled or disable the field located at path. If tooltip is provided, the tooltip for the field will be updated.
    getFieldWidget ( path, default=None )Return the widget located at the given path. If a widget cannot be found, return default. Note: this function returns the actual field object/widget. To get the field's value, one must call getValue() on that object.
    getClusters()Return a list of all avaiable cluster names in Qube.
    getGroups()Return a list of all available group names in Qube.
    getHosts()Return a list of all available hostnames in Qube.
    getJob()Return the job passed in to this submission dialog at launch. The return value will be a dict. If no job was passed in at launch, then an empty dict is returned.
    getUserProperties()Return a list of admin-defined host properties, i.e. those host properties that have been defined via qb[wrk].conf.
    resetParameter( path )Force all fields in the parameter located at path to reset themselves to their default values, as defined in the submit_script 
    setFieldValue( path, value )Update the field located at path with value.

    Field Accessor/Helper Functions

    The dialog's getFieldWidget will return a Field object. Field objects have the following methods:

    getDefault()Return the field's default value, as is defined in the submission script.
    getSubmissionData()Returns a dictionary of used by the internal submitter. Data includes, but is not limited to name, arg, value, and destination.
    getValue()Returns the field's current value. Datatype of the return depends on the opttype of the field - string fields will return strings; int fields will return int, etc.
    isDirty()A field is considered dirty if it's value was changed after initial load.
    isMandated()Returns a tuple of (mandated, level). mandated is True/False. level is is the tier at which the mandate exists.
    resetValue()Set the Field's value back to its startup value. Note: this is the value loaded before preferences.
    setLabel( label )Set the Field's label (not to be confused with the Parameter's label).
    setValue( value, ignore_mandate=True, block_signal=False)

    Set and display the given value in the current Field.

    Arguments:

    • value: the value to set. This should be opttype correct – a field of opttype "int" will expect and integer value whereas a field of opttype "string" will expect a string
    • ignore_mandate: set this value, even if the parameter has a mandated value
    • block_signal: do not notify watchers of the change.

     

    User-defined callback functions

    Callback functions allow the user some control over how fields relate to one another and/or how to populate fields with the results from a "Browse" dialog.

    For example, in the AfterEffects BatchRender submission dialog, when "Creative Cloud" is unchecked, the "CC Year" field is disabled while the "CS Version" fields are enabled.  The "Aftereffects path" field is also updated to use the CS6 path rather than the CC path.  If one were to change any of the CS version fields, then the "Aftereeffects path" field will also change. 

    As another example, when selecting multiple reservations in the Reservations "Browse" dialog, the reservations field is populated with each selection separated by a comma, but when selecting multiple restrictions, each restriction is separated by an OR ('||').

    Function prototype

    Callback functions are defined with the following prototype:

    where:

    • dialog_object is the current dialog.
    • selections are a list of responses from a dialog (if one was used)
    • **kwargs is a place-holder for forward compatibility.

    Return value

    While no return value is required, the return value is written to ArtistView's logs, therefore it is advised the the function return the field's new value.

    Using callbacks

    The submission dialog knows about callback functions through a field or button's "update_function" attribute, i.e. "update_function": function_name.

    Through the dialog_object function parameter, one has access to all fields and their values.

    Example

    Consider the following two parameters:

    The parameter "version" contains a single integer field.  The parameter "path" contains a single path field that defines a watch_list that references the integer field in "version". Because of the watch_list, when the "version_number" field changes, the update function called updatePath will be called.

    Putting it together: when the integer field in the version parameter is changed, because of the path's watch_list, the updatePath function will be called.  That function will get value of the version parameter's int field, and use it to set the path parameter's path field.

    Working example: skeleton.py

    The following submission script defines the skeleton submission dialog.  It is included with your distribution, but is hidden from ArtistView's Submit menu.  You can, however, launch this dialog from the command line as such:

    Skeleton.py is located in your distribution's submit_scripts directory; this is most easily found by launching ArtistView, then going to File > Open Submission Scripts Dir...

    Skeleton.py  Expand source
    • No labels