In this post I’ll compare costs of computing performed in different environments and I’ll try to convince you that Golem is a very competitive solution.

I conducted a little experiment in which I collected and then compared data on costs of performing two chosen Blender tasks. Having an electric energy consumption meter, I was able to directly measure energy intake, and thus cost, of running the tasks on a PC (for two cases: running as a Golem task and natively in Blender). Because I had only one energy meter, I ran the tasks in Golem on only one PC. It shouldn’t affect negatively the measurement’s precision.

I searched the Web for best-rated render farms supporting Blender. In many cases they didn’t work with my simple benchmark, so finally I picked five of them based primarily on me being able to run the benchmark without serious problems:

1. Render Street Quite big (around 3.5THz power) and professional. My personal favourite, nice tooltips result in a great user experience.
   
2. Render Farm.pl I estimate the computing power at 1THz, UX is not bad.
   
3. Turborender Large one (16THz). The overall user experience is very good (apart from language randomly switching to Russian), intuitive UI.
   
4. Render Spell Small (0.6THz) and cheap
   
5. GarageFarmFast, one of the biggest ones (11THz)
   

The pricing is usually given as a 1GHzh cost or as a price of using a single CPU for one hour. It’s worth noticing that number of GHzh required to compute a task may vary from machine to machine, depending on computing unit’s architecture and other features.

Benchmarks

Both my benchmarks were Blender render tasks. The idea was to choose two completely different animations to show that Golem is great in dealing with various cases!

• Cornerstone Centre — 100 frames of an animation by Todd McIntosh, a respected CGI artist and Blender professional. It’s an animation with no complex geometry, consisting of 575 frames. In order to reduce the rendering time, I decided to take only the first 100 of them.

• The well-known Gooseberry Project benchmark — a very challenging animation with a lot of details, which is considered to be a test for big and really powerful systems. The benchmark consists of only seven frames, but each one of them requires a lot of time to render.

The very important thing is how the scene is split when rendering in a distributed system. Most of the farms only support splitting an animation into single frames. In Golem it is possible to split a frame into several subtasks, but for the results to be consistent, I decided to apply the same splitting as in farms.
Splitting a task properly is a little tricky. The problem is that every subtask has to be set up, which includes sending data needed to perform computation, opening Blender and loading the scene. A task which is split into more subtasks is rendering faster (if the number of nodes in a system is sufficient), but the total processing time (understood as a sum of processing times of all nodes) is longer — this is our own variant of
Amdahl’s law.

Results

Cornerstone Centre

As a reference machine I used my desktop PC (Intel® Core™ i5–2400 CPU @ 3.10GHz × 4 and 8 GB RAM). The rendering time was 27h, total electricity consumption around 2.34 kWh, which costs about $0.7 (assuming $0.3 per kWh, more than in most developed countries). The same PC consumed 4.125 kWh in 47,5h while using Golem ($1.24 ).

But what would happen if the Golem test was run not on a single PC, but, let’s say, in a network in which 100 machines like this one were willing to compute a subtask of the benchmark?

Well, it would take around 30 minutes and the electricity cost would still be the same (USD 1.24). Note that the machine used in this experiment is just a regular, a bit old office desktop, not a system optimised for high performance and/or efficiency.

The full results, including render farms, are presented on the chart below.

Gooseberry

The PC used for Cornerstone task wasn’t able to run this benchmark (the scene requires at least 12GB RAM), so this time my reference machine was a powerful Intel core i7–4820K CPU @ 3.7 GHz x 8 and 32 GB RAM.

Obviously, the processing time was longer when using Golem than when scenes were rendered directly from Blender (therefore the energy consumption was also higher). There are a few reasons for this:

Golem starts a new Blender instance for each processed subtask

• because of the security reasons, tasks are computed in a docker container, which may also be slightly slower
• Golem has to do a bunch of different things, like keeping connections with other nodes, sending task resources, verifying computation results, etc.

Note: since Golem allows to split a frame into subtasks, it is possible to render even faster than the calculation shows, if a sufficient number of Golem nodes is involved in computation. We decided to split the task into seven subtasks to mimic the way render farms split their tasks.

What does it all mean?

Our test proves that the cost of computing with Golem is significantly lower than render farms’ fees for the same task, while Golem’s performance might be better than most of the farms and comparable with the fastest (and the most expensive) farms. That of course does not mean that prices in Golem will be that low: they will be as low or high as decided in (automatic) bargaining process between requestors and providers. But what this experiment shows us is that while using Golem, you can achieve performance comparable with the best solutions available on the market and huge producer surplus, i.e. the difference between prices on the market as they are now and the cost of production (which in the case of Golem is equal to the cost of energy).

And this huge gap is exactly where Golem’s first business case fits in.