Versions Compared

    Key

    • This line was added.
    • This line was removed.
    • Formatting was changed.
    Comment: Published by Scroll Versions from this space and version 6.6-3

    ...

    Clusters are designated in the same way as a directory tree; the root is written as '/', and clusters in the root are written as /A, /B, and /C/D (/C is in the root, and D is "belowabove" /C) 

    Defining a cluster (you don't really)

    ...

    The cluster membership of both a Worker and a job is considered when determining the overall priority of a job to run on a particular Worker. The basic rules are: 

    • a worker will run the worstlowest-priority job that is in its own cluster before it runs the besthighest-priority job from any other cluster.
    • if there are no jobs in its own cluster waiting to run, a worker will then run jobs from other cluster in decreasing priority.
    • some or all workers in a cluster can be restricted to only running jobs in their own cluster; this is set with the worker_restrictions parameter
    • a job takes a priority hit when it traverses away from the root.
    • for every cluster boundary crossed when traversing away from the root, the job takes another priority hit.
    • there is no priority hit when traversing towards the root

    So jobs from "other" clusters have less priority than jobs from the same cluster. But there is even a precedence ordering for all the "other" clusters; jobs in clusters nearer a Worker's cluster are considered before jobs from clusters farther away. It "costs" a job in priority to run in clusters other than its own, and the cost increases as the distance between the job's cluster and the Worker's cluster increases. 

    Climbing "up" the tree (toward the root) is usually at no cost. However, a job submitted to /A/B/C has higher priority in /A/B/C, compared to running in /A/B, but that's just because the former case is in its own cluster. It's when a job starts climbing down the tree that it starts to lose priority.

    When a job cannot find any hosts by traversing descending down its tree branch towards the root and has to start going away from the root the "climbing" the tree to find hosts, it loses priority. The more cluster boundaries is has to cross as it traverses away from the rootlevel it has to climb up, the lower its priority. 

    If job1 is submitted to /A and job2 to /B, then they both will have the same priority in /C. (each job had to only go 1 level down)

    If job1 is submitted to / and job2 to /B, then still both of them have the same priority in /C. (again, each job had to only go 1 level down)

    If job1 is submitted to /A and job2 to /B, then job1 jobA will have higher priority than job2 jobB in /A/C because

    • job1 only has to cross 1 cluster boundary away from the root: /A -> /A/C (1 cost in priority)
    • job2 has to cross 2 cluster boundaries away from the root:
      1. towards the root: /B -> / (no cost)
      2. away from the root: / -> /A (1 cost)
      3. away from the root: /A -> /A/C (1 cost)

    ...