Creating Submission Dialogs
Submission UIs are driven by basic Python [ordered] dictionaries. In this section, we will describe how to create and install submission UIs.
There exists a working, well documented example called skeleton.py in the distribution's submit_scripts directory. This directory is most easily found by going to File > Open Submit Scripts directory...
- Creating Submission Dialogs
- Terms used in this document
- Script Location
- Dialog Definition
- Dialog manipulation
- User-defined callback functions
- Working example: skeleton.py
Terms used in this document
- Parameter: A container of fields. A parameter typically defines a label (displayed on the left) and at least one data field, lined up in a row on the right of the parameter container.
- Field: A field holds data that will be used by the submission dialog to create a job in Qube.
- Qube tabs are "stock" tabs. Qube tabs contain all qube-specific parameters/fields, things like job name and worker selection. All Qube tabs and all parameters within them will automatically be a part of every submission dialog.
- User tabs are created by submission scripts. They will contain parameters specific to this submission dialog (and therefore the application being driven by this dialog). All developer-generated content will exist in the user tabs.
Submission UIs are loaded by ArtistView at startup, though they can also be invoked from the command line through the qubeSubmission[.exe] executable. For a submission UI to be visible in either location, it must live in a location about which ArtistView knows. Default locations for submission scripts are:
User-defined submission scripts can be defined in ArtistView preferences, under the "Submission" tab:
Submission scripts can be easily found by going to File > "Open Submission Script Directory..." in ArtistView.
The most basic submission dialog is a python dictinoary. It must be defined in a .py file within one of the submit script directories and assigned to a root-level variable called
dialogs. The dictionary must be keyed on some submit-script name with a value that, itself, is a dictionary with at least one key called "parameters" who's value, again, is a dictionary. For example:
Multiple dialogs can exist in the same submission script by simply adding more keys at the "my_dialog" level.
Dialogs can be defined as basic dictionaries, though basic dictionaries have no sense of order, so when order is important, use an OrderedDict. Depending on the version of Qube you are using, you may need to "
from util.av_collections import OrderedDict" at the top of your script to have access to OrderedDicts. OrderedDicts are accessed exactly like dictionaries, though they are formed more like functions.
For example, if this is how one creates a dictionary called 'd':
...then this is how one creates an OrderedDict called 'd':
The primary difference is that when we iterate through the OrderedDict, the keys will maintain order, whereas a basic dict does not guarantee this behavior.
Dialog Top Level Attributes
Submission UIs have the following attributes that are defined as keys at the top level (in the example from the last section, the "top level" of the submission UI is the same level as "parameters").
|display||Display name - this shows up in menus and title bars. If not provided the script's key name will be used.|
|shortName||Short display name - this will show when there is little space (i.e. tab). If not provided, the display name will be used.|
|prototype||The job's prototype, i.e. |
|help||Content for help button. Can be plain text or basic html. The final product will be html wrapped in <pre> tags.|
|hasRange||Should we present a frame range widget? True/False (if omitted, then False)|
|canBatch||Should we present a range execution widget? True/False (if omitted, then False)|
|canPadFrames||Should we present a frame padding widget? True/False (if omitted, then False)|
|rangeRequired||Must the user provide a frame range? True/False (if omitted, then False)|
Should we preset thread controls? "all"/"specific"/False (if omitted, then False).
|preDialog||Function to be run before the dialog is displayed|
|preShowValidate||Validation function to check data sent to the submission dialog. If validation fails, the dialog will not be displayed.|
|postDialog||Function to be run immediately after the dialog is displayed|
|preSubmit||Function to be run immediately before the job is submitted|
|preSubmitValidate||Validation function that is run prior to submission - designed to validate submission options. If validation fails, the job will not be submitted.|
|postSubmit||Function to be run immediately after the job is submitted (having access to the newly submitted job)|
|category||Category in the Submit pulldown menu in the main interface|
|parameters||Placeholder for our OrderedDict of parameters.|
|hidden||Should this submission dialog show up in the main interface's Submit menu?|
Dialog Pre/Post Functions
A dialog can make use of setup, teardown, and validation functions. These functions are defined at the top level of the dialog (as mentioned above). The functions are:
|Function Name||Arguments / Return||Description|
This function is designed to do final setup of the dialog prior to display.
For example, for Maya submissions, the user may want to change the current renderer from the setting that is stored in Maya to one of the other options that are available in the scene. During submission, all possible renderers are encoded into the passed-in job object which preDialog parses and adds each item to a drop-down menu in the submission dialog.
This function is intended to provide a mechanism to validate settings that may be necessary prior to the dialog being displayed.
For example, if a scene has not yet been saved, or if a setting within the scene will prevent it from rendering. In those cases, the developer would return some sort of a message from this function which will then be visible to the user who can then solve the problem and try their submission again.
PostDialog is run after the user clicks submit, but before the cmdline or agenda has been created. It is designed to allow the developer to manipulate data prior to the creation of the the cmdline and agenda.
For example, AfterEffects (AE) can render out to either an image sequence or a movie file. An image sequence can be distributed across many computers (each getting a frame or group of frames), whereas a movie file can only be rendered by a single machine (an AE limitation, not a Qube limitation). The AE submission dialog's postDialog determines the output type and changes the instance count to 1 and moves from rendering multiple frames to rendering a single partition (all frames on one agenda item) when the output type is a single movie file.
This function is designed to allow access to manipulate job data prior to submission. Unlike postDialog, preSubmit is run after the agenda has been created and after the command template has been formed - in other words, It is run after the job-to-be has been fully realized.
For example, Cinema 4D does not log the path to the file it has written - as such, Qube cannot determine the output path by parsing the outputlog; instead, the output paths are pre-populated into each agenda item prior to submission. This is done by preSubmit.
This function is the final validation step prior to submission. It allows the developer to, e.g., double check any generated values, check the existence of output paths, check permissions, verify the correct options were chosen, etc.
For example, within the last few AfterEffects (AE) releases, when creating an output from a render queue, the "Save in subfolder" option is turned on by default. This will tell AE to create a subfolder in the output directory that is named after the composition. When rendering through AE's UI, this directory is created if it does not exist, but when rendering via the command line, aerender will NOT create the directory, thus causing the render to fail. Qube works around this by using preSubmitValidate to verify that the path exist prior to submission.
PostSubmit is run after the job has been submitted to and accepted by the supervisor. The passed-in "submitted" argument contains a list of jobs that were accepted by the supervisor. When submitting one job at a time, "submitted" will be a list of one job that is fully formed. When submitting more than one job at a time (in the same submission dialog - which would be rare), then the return will be a placeholder job, with only the job id and submitted time values populated. This function allows the developer to perform any sort of clean-up or notification about the job's existence.
For example, after a job has been submitted, one might want to update a production tracking system, or send an email, or clean-up local temporary storage, etc.
All parameters are described by python dictionaries, keyed on the parameter name. Each parameter defines several top-level attributes and a dictionary of field names keyed on the field name. For example:
The above code block would define a parameter with both the internal name and the display name of "simple_path" but contains no fields.
If "fields" is a top-level attribute of a parameter, then other available top level attributes are:
|fields||An [Ordered]dictionary of parameter fields. See next section for more information|
|hide||If true, the parameter cannot be seen by the user|
|label||The name displayed in the UI. If not included, the key name of the parameter's dictionary will be used|
|required||If True, then the parameter label will be red and the job will not submit while any parameter in the field is empty|
|tooltip||Text displayed when the user hovers over the parameter. Fields can optionally provide their own tool tips in addition to that parameter's tool tip.|
|tab||The tab on which the parameter will reside. Excluding "tab" will cause the parameter reside in the main lower tab.|
Parameters are containers for fields. Fields hold the actual data that will eventually be submitted to the farm. Like all parts of the submission UI, fields are python dictionaries, keyed on the field name. For example:
The above code block will define a "path" field called "simple_field" that has no label.
If "opttype" is a top level attribute of a field, then other available top level attributes are:
|always_use||Always include this field and its data in the submitted job, even if they are left blank/unchagned by the user.|
|arg||The command line argument to which this field should map. "Arg" is only necessary if mapping directly to a command line argument.|
|default||The default value for his field. Note: this is not the same as "value". "Default" means "this is the value the application expects, therefore the application will operate the same if this value is excluded." When using a default value, the value will be displayed in the submission UI, but will only be passed to the job if changed to a non-default value. There is never a need to use both a "default" and a "value".|
|destination||Where this key/value pair should be included in the job being submitted. For example, "Priority" has a destination of "job", as it will live at the top of the job object. "Maya executable" will have a destination of "job.package", as it will live in the job's package. Options that are used only by callback functions may not need to live in the job at all – those would have a destination of None.|
|destination_type||Currently, this can either only be excluded or set to the Python type "dict". If set to dict, then the field value is expected to be a python dictionary. This is only valid for opttype "combo" fields.|
|label||If provided, the field will get its own label to the left of the field. As an example, see "Farm Concurrency" in any submission dialog. "Instances:" and "Max:" are field labels.|
|options||opttype-specific options. See below for more information.|
|opttype||The type of field to use. See below for more information.|
|value||The value that should be displayed in this field when the dialog is launched. Unlike "default" values, a "value" will always be passed to the job, even if not changed. There is never a need to use both a "default" and a "value".|
|update_function||Callback function that is fired any time a field in this field's watch_list is updated.|
|watch_list||a list of fully qualified field names that determine the value of this field. For example, in a "Maya Jobtype" submission dialog, if the user clicks the "Each instance uses all cores" option in the Qube parameters, then the Maya parameter "Threads" must be set to 0 to instruct Maya to use all available cores on the worker. If the user unchecks "Use all cores", then changes the "Threads per instance" value, then the Maya parameter "Threads" must be set to match the "Threads per instances" value. In this case, the Maya "Threads" parameter would have both the "Threads per instance" and "Each instance uses all cores" fields in its watch_list.|
Field opttypes and options
All fields are defined the same way, yet not all fields are the same. For example, there's no reason to define a number range for a string field, and there's no reason to define default text for an integer field. All field opttypes have their own set of options
|opttype||definition and options|
A number. The presented widget will only accept a number and will provide up and down arrows to change the value
A single-line string field. The presented widget will be a simple, single-line text-entry field.
Multi-line string field. The presented widget will be a multi-line text-entry field.
|combo / combo_path|
A combo box (or drop-down) field. The presented widget will be a typical combo box, without icons, with a down-arrow on the right that, when clicked, will show all available options.
"combo" and "combo_path" are identical with the exception that the value passed to the job from a combo_path will be treated as a path (being automatically converted by the worker at render time, if requested), whereas a combo's value will be treated as a string.
An editable table of data. The presented widget will be a table of text-entry fields.
A calendar field. The presented widget will look like an editable combo field, but will present a calendar when the down-arrow is clicked.
Submission dialogs must be able to be updated programmatically while being used. In addition to setup, for example, it is quite common for rendering options to be mutually exclusive or for one option to effect another. In those cases, the developer must be able to both monitor specific fields for changes and change fields as necessary. We will describe the various methods for doing so in this section.
Parameter and Field Addresses
In order to make changes, one must be able to find the field and/or parameter of interest. Parameter and field addresses (aka path) follow the form:
|From outside the submission dialog|
|From within the submit script|
For example, the "Farm Concurrency" parameter has two fields, labelled with "Instances:" and "Max:". The parameter is defined as such:
To gain access to the field labelled "Instances:", we could use the address
To gain access to the field labelled "Max:", we would use the address
Dialog Accessor/Helper Functions
The dialog object (that is passed into most functions that are run by the submit script) has a few helpful functions for accessing widgets and/or their values.
|archiveJob( job )||When passed in a qb.Job, this function will display a "File Save" dialog box, allowing that job to be written to disk as a binary Qube job archive (.qja)|
|enableField ( path, enable=True, tooltip=None)||Enabled or disable the field located at |
|getFieldWidget ( path, default=None )||Return the widget located at the given |
|getClusters()||Return a list of all avaiable cluster names in Qube.|
|getGroups()||Return a list of all available group names in Qube.|
|getHosts()||Return a list of all available hostnames in Qube.|
|getJob()||Return the job passed in to this submission dialog at launch. The return value will be a dict. If no job was passed in at launch, then an empty dict is returned.|
|getUserProperties()||Return a list of admin-defined host properties, i.e. those host properties that have been defined via qb[wrk].conf.|
|resetParameter( path )||Force all fields in the parameter located at |
|setFieldValue( path, value )||Update the field located at |
Field Accessor/Helper Functions
The dialog's getFieldWidget will return a Field object. Field objects have the following methods:
|getDefault()||Return the field's default value, as is defined in the submission script.|
|getSubmissionData()||Returns a dictionary of used by the internal submitter. Data includes, but is not limited to name, arg, value, and destination.|
|getValue()||Returns the field's current value. Datatype of the return depends on the opttype of the field - string fields will return strings; int fields will return int, etc.|
|isDirty()||A field is considered dirty if it's value was changed after initial load.|
|isMandated()||Returns a tuple of (|
|resetValue()||Set the Field's value back to its startup value. Note: this is the value loaded before preferences.|
|setLabel( label )||Set the Field's label (not to be confused with the Parameter's label).|
|setValue( value, ignore_mandate=True, block_signal=False)|
Set and display the given value in the current Field.
User-defined callback functions
Callback functions allow the user some control over how fields relate to one another and/or how to populate fields with the results from a "Browse" dialog.
For example, in the AfterEffects BatchRender submission dialog, when "Creative Cloud" is unchecked, the "CC Year" field is disabled while the "CS Version" fields are enabled. The "Aftereffects path" field is also updated to use the CS6 path rather than the CC path. If one were to change any of the CS version fields, then the "Aftereeffects path" field will also change.
As another example, when selecting multiple reservations in the Reservations "Browse" dialog, the reservations field is populated with each selection separated by a comma, but when selecting multiple restrictions, each restriction is separated by an OR ('||').
Callback functions are defined with the following prototype:
dialog_objectis the current dialog.
- selections are a list of responses from a dialog (if one was used)
- **kwargs is a place-holder for forward compatibility.
While no return value is required, the return value is written to ArtistView's logs, therefore it is advised the the function return the field's new value.
The submission dialog knows about callback functions through a field or button's "update_function" attribute, i.e.
Through the dialog_object function parameter, one has access to all fields and their values.
Consider the following two parameters:
The parameter "
version" contains a single integer field. The parameter "
path" contains a single path field that defines a
watch_list that references the integer field in "
version". Because of the watch_list, when the "
version_number" field changes, the update function called
updatePath will be called.
Putting it together: when the integer field in the
version parameter is changed, because of the path's
updatePath function will be called. That function will get value of the
version parameter's int field, and use it to set the path parameter's path field.
Working example: skeleton.py
The following submission script defines the skeleton submission dialog. It is included with your distribution, but is hidden from ArtistView's Submit menu. You can, however, launch this dialog from the command line as such:
Skeleton.py is located in your distribution's submit_scripts directory; this is most easily found by launching ArtistView, then going to File > Open Submission Scripts Dir...