Versions Compared

    Key

    • This line was added.
    • This line was removed.
    • Formatting was changed.

    ...

    There are a lot of variations that can occur here, but the important thing to remember is the distinction between Instances and Agenda Items - Agenda Items (also commonly referred to as frames when rendering) are the granular list of things the job wants to accomplish, while Instances designate the number of machines the job is run on. (The simplest example is where the number of Instances equals the number of Agenda Items, but sometimes that's not the most efficient way to do things)

    Assigning a Job

    Now the Supervisor has a job, and it goes looking for a Worker to run it. It does this by taking into account a lot of different information about the job, the workers, the current state of the farm, etc. This is a greatly simplified description of that process.

    The Job Slots (or Process Slots) for a Worker determine how many processes a given Worker can run at a given time. The Job Slots have no direct relation to the number of processor cores on a given machine. It is often desirable when using a multithreaded renderer like Maya to have the Worker run 1 instance (or process) at a time. That process will in turn utilize all processor cores for the rendering. If wanting to run many single-threaded processes that require little resources, one may want to set the Worker to have 4 or 8 Job Slots available. 

    Every job in Qube! is assigned a numeric priority. Priority 1 is higher than priority 100. This is similar to 1st place, 2nd place, 3rd place, etc. The default priority assigned to a job is 9999. The priority is used to help determine when and where (on which Worker) a job will run. But there is more to it than that.

    ...

    Jobs can be submitted with restrictions, such as "only run on OS X". Workers can also have restrictions on the number or kinds of jobs they will run.

    Qube! has the concept of clusters. A cluster is a group of machines that is given a name. Clusters have a hierarchy, so that if a job cannot be run in one cluster, it may be able to run in a parent cluster. The most common example of this is show/sequence/shot. If the appropriate resources are not available in the "shot" cluster, the job may be assigned to the "sequence" cluster.

    Reservations

    Restrictions

    Job States

    See Also

    How to use clustering for workers
    How clustering affects priority and worker selection for jobs
    worker_cluster 
    supervisor_default_cluster
    client_cluster

    Workers have resources, such as CPUs and memory, that jobs may specify as requirements. The facility may also have resources, such as software licenses, that jobs my specify as requirements. The Supervisor looks for Workers that have those particular resources, and then reserves slots on them for the job(s) it wants to assign.

    Running the Job

    Taking all the above into account, the job is assigned to a specific Worker. As that happens, there may be some path translation (for example, if the job was submitted from Windows but picks up on a Linux machine). The job's state will be changed from "pending" to "run". Assuming the job runs successfully, the state will change to "complete" and the logs it wrote will be available for viewing in both WranglerView and ArtistView. The images it created (if any) will also be viewable in those UIs. The Worker will advertise itself as available, and the Supervisor will look for another job to assign to it.

    Qube! also has a robust understanding of dependencies, so that jobs can be chained together, with the start of one or more jobs depending on the completion of one or more other jobs.

    See Also

    Qube! Job Reference