Developer Tech News #2
Welcome to our second installment of our weekly developer update blog. As we said last week, this blog is here to inform our technical followers about the progress we are making towards a full release of Golem’s software.
IMPROVING OUR CONTINUOUS INTEGRATION
Golem has maintained an extensive suite of unit and integration tests whose aim has been to ensure flawless operation and prevent regressions. Computers are fast, never tire and are merciless, so they are perfect for discovering mistakes.
Golem’s Continuous Integration suite is now being extended so it contains static analysis tools. These include pylint, pycodestyle and mypy.
The first one is a long-developed and comprehensive linter for Python code. Its importance originates mainly from the nature of the Python language, which means that many errors cannot be caught until runtime. Pylint tries to alleviate this problem, trying to find those errors before any executable has been started.
The second one is a style enforcement tool. Consistent code style is so important in big projects like these and consistency makes it much easier to find your way in the code.
The last one is a brand new widget, which is younger than Golem itself. The dynamic nature of Python can become a problem when maintaining a huge codebase, which gave birth to the Python type annotations, introduced in PEP-484. Mypy is a tool for verifying those. It’s a very powerful tool for catching mistakes and documenting code. While the current scope of this linter in our project is quite limited, it will become more and more useful as type annotations are added to the project sources.
The static analysis tools will prevent new errors and style violations from sneaking into new/changed code.
DEFEATING JSONPICKLE
As many of you may already know, the most common serialization format in Python is pickle, however it has one large and inherent problem. As Marcin Mielniczuk, one of our developers explains, unpickling untrusted data leads to arbitrary code execution. Secondly, pickle is largely unportable which makes its integration with other programming languages a problem.
In many cases, Marcin says, it’s enough to do the serialization with the standard JSON module. It works whenever the type is “primitive”. However after some research, the team has decided to replace those processes with safer alternatives wherever possible to rule out any possibility, however hypothetical, that a Golem instance is fed by unsafe, malicious data. jsonpickle problem originally consisted of six occurrences and four of them are already fixed.
FIXING GOLEM INSTALLATION SCRIPT, WRONGLY ENCODED ETHEREUM ADDRESS
We’ve had some minor problems with building installers mainly on Windows and MacOS. To fix this, and be able to prepare installers and binaries that work smoothly, they have made some changes in building scripts, mainly in golemapp.spec. That has involved changing the options passed to PyInstaller to make sure that Golem will use a proper version of libraries — install_script.iss — which are responsible for building the installer for Windows OS.
Doing this has been really important for Golem, because it allows us to distribute the newest version of our software based on Python 3. (Adopting Python 3 eliminates a lot of other problems).
Paweł Peregud, our Senior Software Engineer also reports that we have fixed an old regression that caused the Ethereum address of a given node to be improperly displayed in Golem’s UI. Before the fix, these same mangled addresses were being passed to other network participants (but fortunately weren’t passing correctness tests so didn’t cause further problems except some confusion).
As you can see we’ve achieved a lot this last week, which is even more remarkable because August 15th was a national holiday in Poland. After a refreshing break I can already see that the team is roaring ahead with progress so we’ll have more to add by the time of Developer Tech News #3!
DEVS TEAM