What’s new in Golem v0.5

Golem Project -

[edit: Please note that this version is outdated. Visit our github for the up to date installer of Golem.]

With the recent announcement of version 0.5 for macOS, Ubuntu, and Windows, we would like to bring you up-to-date with the most significant changes in Golem.

Ethereum light client

There is no denying the fact that blockchains are the next big thing. Also, literally big — up to tens of gigabytes of data which takes hours to download. Thankfully, now we have tools at our disposal to make the synchronisation process significantly faster, keeping the blockchain data to a necessary minimum.

The new and shiny version 0.5 is equipped with geth (the Ethereum client written in Go) working in a light client mode. When you type golemapp in your terminal, it boots up a geth light protocol node and waits few only minutes for initial block synchronisation. This way we assure that the required blockchain parts have been downloaded and you’re ready to roll with Golem. Considering that the application requires some data that is not present in the downloaded part of the blockchain, the light protocol will query the network and download those parts on demand.

Naturally, the light client mode is not the remedy for all synchronisation and resource problems, but in Golem’s case it comes pretty close. Personally, I am strongly for trading few more CPU cycles and a little more bandwidth to be able to jump (almost) right in, and spend those extra few hours on computation instead.

tGNT

Although tGNT has been mentioned in a previous blogpost, this is the place for a quick reminder. Golem o-point-five uses test GNT (or simply tGNT) as remuneration for providers. There are few things to remember about test tokens:

  1. tGNT does not hold any real value.
  2. tGNT is not correlated with your real GNT.
  3. tGNT is fun and everyone should have some.

Because of (3.), everyone in the Pirx testnet automatically gets a modest amount of test tokens from our faucet, just to start working with Golem. You will also need some test Ether, but do not worry, there is a faucet for that. Remember that test Ether does not hold any real value either.

If you dig a little into Golem’s source code, you could see that the current M.O. is payment batching. This means that your Golem node will send multiple payments at once. You can take a good look at all of your testnet transactions on etherscan.io. We encourage you to report any problems you are experiencing with payments on our Slack channel.

macOS

macOS support is not exclusive to version 0.5, but still it has been a major step since the 0.3 pre-alpha. Running Golem on Apple’s operating system was possible due to taking these two major steps:

  1. Switching from Qt4 to Qt5 (the user interface).
  2. Implementing support for the xhyve hypervisor.

Migration to Qt5 was necessary mainly due to Homebrew’s PyQt4 installation process no longer being supported on macOS Sierra. Being more precise, building PyQt4 for Python has become troublesome. Either way, few custom PyQt4 builds and even more tests later, it was clear that Golem required some adjustments to run it’s previous GUI on OS X. It was a high time to jump to the next version of Qt and at long last introduce a macOS version.

As for the second point — xhyve is a virtual machine hypervisor built on macOS’ native Hypervisor.framework and basically a prerequisite for Docker on Mac. xhyve runs Docker’s small Linux distribution, which is in turn responsible for firing up Docker images. In short, assigning CPU cores and the amount of RAM to a Docker image implies a change to Docker VM’s parameters. Currently, that is Golem’s responsibility.

Resource sharing

Version 0.5 has been equipped with a new resource-sharing tool, tentatively named HyperG. It’s a custom application built on top of hyperdrive, a real time distributed file system and a beating heart of Dat project. Readers who are not familiar with the Dat sharing tool may read up on it here. To be clear, the introduction of HyperG does not mean that we are leaving IPFS behind. So why the unexpected change?

During our pre-alpha tests we noticed that previous IPFS versions were opening a fair amount of TCP connections quickly. As a result, our office and home routers couldn’t keep up with connection tracking, as NAT tables were overflown and connections were being dropped. In turn, we encountered problems with resource distribution in Golem. The IPFS team has worked a great deal on this issue since then and the situation has significantly improved. However, that day we considered using other solutions. One of the most interesting was the Dat project.

Dat operates in a different manner than IPFS. There are opt-in BitTorrent-like swarms, and file transfers are generally point-to-point. Also, Dat does not try to persist immutable data on its network, and in fact allows the shared archives to be modified later on. Overall, it appeared to be a more lightweight solution.

We did some internal testing where both solutions were performant under different NAT configurations. Dat has proven to be less I/O intensive, and used significantly less bandwidth. We should remember that both applications operate in a different manner and serve slightly different purposes. In any case, that very same day, we began working on a prototype based on Dat that would be closer to our use case, and now it’s being tested on the Pirx testnet.

HyperG bodes well for the future, and we plan to continue the development. Unfortunately, the current version lacks port-forwarding compatibility and could use improvements in memory management. HyperG will see a new release at some point during the lifetime of the Pirx testnet, and will probably appear under a more memorable name as well.

Optimisations

Golem has seen optimisation in its networking code, specifically in encryption, signing and verification of network messages. This was a minor patch to the pydevp2p library, which provided a static initialisation context for libsecp256k1, instead of creating a new one each time a message needed to be processed. Testing and profiling has shown that Golem’s CPU usage dropped significantly on all supported platforms.

You might have also noticed that Golem is more performant on Windows. We have achieved this by using the Windows’ native event notification system (I/O Completion Ports) in Python’s twisted framework. Thanks to IOCP, all asynchronous code should now be more robust. This optimisation had the most significant impact on networking and communication between Golem and the UI.

Versioning

Prior to the announcement of the Pirx testnet, Golem was getting back on track with versioning. This process begun with the first (since the early alpha stages) macOS version, more widely known as 0.4.0. You might have spotted few macOS nodes popping up in the vanilla test network.

Since the bump to 0.5.x, the patch version “x” has been incrementing steadily, as we fix arising issues. Unfortunately, you can’t find any change logs for these releases. As soon as the testnet version of Golem gets out of the bug-fixing stage, we plan to standardise our release process.

Last, but not the least

Finally, as Grzegorz Borowik mentioned in his last blog post, we have built Golem installers for macOS, Ubuntu and Windows. As we plan to support other major Linux distributions, we want to take it one step at a time. We would love to hear your feedback and if you find any bugs or simply feel that the installation process feels a little off, please let us know on our Slack channel.