Proposal for an alternate LoadBalancer Algorithm for Crowdrender based on a multicolored checkerboard

Introduction

Crowderender is an addon for Blender 2.8 that distributes the render process across multiple computers by assigning areas of the render image ( tiles ) to each computer. The goal is to speedup the render process in total. The assumption is that the render load can be distributed ( efficiently ) and be faster in sum than on a single computer.

The distirubted process involves several steps and latencies ( bottlenecks ):

* The Load Balancer uploads a blender file to each computer – this adds latency to the overall process due to networkbandwidth and latency

* The Load Balacner coordiantes tiles for every computer – every computer get's some tiles assigned to render based on a certain metric or assumption

* Upon finishing the task each computer uploads the rendered tile to the Load Balancer

* The Load Balancer composes the individual tiles to form the final rendered imaged

Implications

It turns out that the LB has no good way of determining or guessing how to distributed the tiles and is therefore inefficient.

A heuristic approach would not improve the overall efficiency as we need to consider the time needed for training the LB.

Also each scene and each tile have different complexities and efficiencies on the computers cpu/gpu.

In short we are dealing with a typical case of a bounded touring complexity problem.

Current example

Assuming we have 2 or more computers involved in the distributed rendering.

Each computer would get roughly 50% of the image. These 2 workloads would finish at very different times and so 1 of the 2 nodes will be idling and the overall rendering time will be worse than on a single computer.

Proposal

Use a multicolored checkers board algorithm where every tile gets potentially dispatched to a different computer.

Split into many tiles ( 4, 8, 16 …. ) and dispatch each tile to an idle computer.

After a tile has been dispatched, rendered and sent back the LB just sends the next tile in line to the any idle node. After all tiles have ben sent back the LB does the final composition.

Efficiency

The LB would not need to determince performance characteristics for the tile distribution. Network latencies and cpu/gpu characteristics would average out during the course of the network rendering process.

Cheers