Versions Compared

    Key

    • This line was added.
    • This line was removed.
    • Formatting was changed.

    ...

    Clusters are designated in the same way as a directory tree; the root is written as '/', and clusters in the root are written as /A, /B, and /C/D (/C is in the root, and D is "abovebelow" /C) 

    Defining a cluster (you don't really)

    ...

    The cluster membership of both a Worker and a job is considered when determining the overall priority of a job to run on a particular Worker. The basic rules are: 

    • a worker will run the lowestworst-priority job that is in its own cluster before it runs the highestbest-priority job from any other cluster.
    • if there are no jobs in its own cluster waiting to run, a worker will then run jobs from other cluster in decreasing priority.
    • some or all workers in a cluster can be restricted to only running jobs in their own cluster; this is set with the worker_restrictions parameter
    • a job takes a priority hit when it traverses away from the root.
    • for every cluster boundary crossed when traversing away from the root, the job takes another priority hit.
    • there is no priority hit when traversing towards the root

    So jobs from "other" clusters have less priority than jobs from the same cluster. But there is even a precedence ordering for all the "other" clusters; jobs in clusters nearer a Worker's cluster are considered before jobs from clusters farther away. It "costs" a job in priority to run in clusters other than its own, and the cost increases as the distance between the job's cluster and the Worker's cluster increases. 

    Climbing "up" the tree (toward the root) is usually at no cost. However, a job submitted to /A/B/C has higher priority in /A/B/C, compared to running in /A/B, but that's just because the former case is in its own cluster. It's when a job starts climbing down the tree that it starts to lose priority.

    When a job cannot find any hosts by descending down its tree branch traversing towards the root and has to start "climbing" the going away from the root the tree to find hosts, it loses priority. The more level it has to climb upcluster boundaries is has to cross as it traverses away from the root, the lower its priority. 

    If job1 is submitted to /A and job2 to /B, then they both will have the same priority in /C. (each job had to only go 1 level down)

    If job1 is submitted to / and job2 to /B, then still both of them have the same priority in /C. (again, each job had to only go 1 level down)

    If job1 is submitted to /A and job2 to /B, then jobA job1 will have higher priority than jobB job2 in /A/C because

    • job1 only has to cross 1 cluster boundary away from the root: /A -> /A/C (1 cost in priority)
    • job2 has to cross 2 cluster boundaries away from the root:
      1. towards the root: /B -> / (no cost)
      2. away from the root: / -> /A (1 cost)
      3. away from the root: /A -> /A/C (1 cost)

    ...