The charting framework allows the QubeWranglerView to visually display historical and current information about the renderfarm and the render jobs. It gives you insight on how your hardware and software are used – and what is needed to meet your rendering needs. It will give you concrete information instead of having you rely on guesses to best manage your studio resources and infrastructure. Data for these charts comes from the Data Warehouse.
This information can display totals along with drilling down be filtered and categorized by individual users, job types, accounts/projects, clusters, and job names (using wildcards). The "Top 5" allows you to see the most influential individual items in a list at a glance without the clutter of all items.
It will keep this historic data and can be plotted across the last 24 hours all the way to years back. You can use the date range presets for today, yesterday, last 24/36 hours, last 3/7/30 days, or put in any custom range. It will provide chart totals along with drilling down to individual items.
There are currently 6 charts initially deployed within the charting framework.
- Running Frames – Displays how many frames (or tasks) are currently running on the render farm. Specifying on a per-user, per-cluster, or per-account, basis lets you see the most influential individual items.
- Pending Load – Shows the number of frames (or tasks) in the queue and when they were submitted. Use this to see patterns in when the most frames are submitted or when there are lulls.
- Farm Utilization – Displays render farm Worker slots currently running versus the total available. See if the farm is fully utilized.
- Farm Size – Displays separate data lines for Worker slots that are active, locked, and down. See how large the farm currently is and how large it can be.
- Server Dispatch Efficiency – Shows the ratio between running instances and agenda items. Less than 1:1 is not good, and less than 85% indicates a problem. See below.
- Global Resources and Licenses – Charts the use of global resources over time. Additional resources can be added and tracked - see Global Resources.
These charts can be printed or saved as images for inclusion into reports.
Charts are not enabled by default. You enable them in Preferences, and then you will see Chart View as one of the options.
Server Dispatch Efficiency
The dispatch efficiency is the ratio of running work items to running instances. When the dispatch efficiency starts to slide below 85% for anything more than 1 or two sample periods, it's usually indicative of a performance bottleneck in your Supervisor. The most common cause is MySQL getting blocked on disk I/O, so it's not MySQL itself that's usually the issue, it's how fast it can write to disk that's the problem.
To be more specific: Workers communicate with the supervisor on one port, and clients and API requests use a 2nd port. The client port is always checked before the worker port, so user interaction gets priority.
When the supervisor starts to get behind (no idle supervisor processes to service an incoming request), client & API connections will wait as long as it takes to get serviced, but Worker connections are sent a rejection message, in which case the Worker will go into a wait loop and retry in a certain amount of time (which increases with each consecutive rejection).
One type of a worker request is to fetch the next agenda item. When this request gets rejected, the instance on the worker is still running, but there is no running agenda item. So you end up with more running instances on the farm than there are running frames; the ratio of runningFrame / runningInstances is less than 1:1. You can see this in the chart, or you can run the qbsummary command and look at the running Subjobs vs the running Work totals.