Golem’s mid-term goals — in detail
Building Golem is an incredibly rewarding experience, we learn new things every day, but at the same time, constructing something that has never been done before is unpredictable and tricky.
Launching Brass Beta into mainnet, for instance, was not envisioned in our first plan. But we understood that in order to progress, we needed to go out of our comfort zone — that is, obviously, testnet and the lab environment, and see how Golem behaves in the wilderness of mainnet.
Now that we accomplished this, that we introduced 2 major updates after that first Brass Beta mainnet launch and a couple of patches, we sat down with our ever-growing team and planned differently. We are not reinventing the wheel but dismantling it and tracking the progress of each piece, while we never stop working and progressing, sticking to our core values and premises.
Golem and Ethereum are evolving, new challenges and solutions pop up every single day. Consequently, we decided to be realistic and honest, and this “roadmap revamp” in Kanban format is aimed to communicate our mid-term goals better. Our whitepaper and original roadmap are not replaced. Goals are planned differently and in a more mindful manner.
To say it differently, this mid-term goals kanban is a holistic approach to our “Golem Masterplan” which we started back in 2014.
Now, let’s get to explaining the main elements of our mid-term goals roadmap. You can follow the paragraphs below, and relate them easily to the columns and cards of the new Golem mid-term goals kanban, which was presented yesterday via livestream :
SGX /Graphene-ng
Golem’s integration with Intel SGX technology is formed by several steps and will achieve crucial goals for our future development.
We consider SGX one of the most promising technologies aimed to solve a wide range of challenges both within our project and the wider blockchain ecosystem. Hence we decided to make it a core component in Golem and ship it for developers across the world seeking for solutions in the verification field. The Intel SGX solution was not developer friendly so the team together with ITL has been working on this, and developing our own solution, in, of course, an open-source fashion.
Once we achieve this milestone, meaning having Graphene-ng up and running in Golem, this will enable features like:
- Local verification will be unnecessary, due to the fact that the whole computation will take place inside a trusted environment.
- Since using SGX introduces some overhead, we may also let the provider optionally compute everything without SGX. Then the provider can perform the verifications via SGX, which will be more cost-effective at the same time, and streamline the proving of correct computations.
- We will be able to decentralize the Concent service. Potentially any provider with SGX can behave as an extra verification service as you don’t need to trust the provider in this case.
As a first step, we will start by making Graphene-ng production-ready and then be implementing the Blender use-case to run fully in SGX.
Brass
We continue to maintain and support Brass Golem as a mainnet product and provide necessary changes to address the most important pain points and introduce new but already planned features. The most significant improvement to happen soon, will be the introduction of the Concent Service on mainnet, a milestone we are eager to present to our users.
As mentioned in our last “Golem’s Guide to trusted computations”: “If we can find a generic approach for this class, it would be easily applicable to deterministic computations or computation for which PoW exist (even though PoW can be used as well).”
Brass Golem will gradually evolve towards Clay Golem. Through a sequence of improvements, changes and additional functionality enhancements, which will be managed by Clay team and merged into Brass once they’re deemed mature and robust enough, we’ll be building Clay Golem on top of Brass Golem.
Once all features planned for Clay have been developed, our flagship product will become Clay and thus, the new default base of the project, which will be continually under development.
Clay
Clay aims to be a major step forward for Golem, focused on the improvement of Brass’ architecture and enriching its feature set. This next milestone is not planned as a single update, but rather as a series of gradual improvements, released over time.
The most notable introduction is the NVIDIA container runtime for Docker. Users will now have the power to execute CUDA applications on their Golem nodes, and also put their GPUs to work for rendering on Blender. This feature will open the door for machine learning and other type of GPU-targeted computations. Please note that NVIDIA container runtime for Docker is only available on Linux systems.
In addition, the Clay team will be working on introducing new computation environments, i.e. Virtual Machines, and other sandboxed environments. With the introduction of the Task API, an evolution of the currently available programming interface, developers will have wider flexibility in creating new types of applications to run on top of Golem.
The ongoing work will upgrade gradually Brass’ architecture through the decomposition of its components in the form of isolated processes, a better orchestration of internal events with message queues and improved application state persistence.
Clay is additionally set to bring improvements within Golem’s network layer. For instance a new networking stack, written in Rust, a language that is being adopted widely by blockchain developers and earning recognition for its speed and safety. The most notable changes are related to NAT traversal and relaying, but also protocol multiplexing, which eliminates the necessity of using multiple ports by Golem. Clay will also aim to replace HyperG (current file transfer tool), improve on discovery and RPC protocols.
RnD
The goal of the RnD team is to define and explore marketplace/marketplaces for Golem. The first stage will be dedicated to the improvement of the Brass Golem economy, preserving its transactional framework. This includes an upgrade of the local reputation system and the market behaviors of nodes.
At a later stage, we plan to define a new marketplace for Clay Golem, which won’t be necessarily evolved from Brass Golem. We consider we have absolute freedom to define it exactly the way we need, including modifications of the most core features like the Golem protocol and the transaction framework. Depending on the needs for this stage, we may require solutions like distributed reputation, anti-sybil attack systems, identification systems, and Concent-like services, etc. The purpose is to find the ideal distributed P2P marketplace for future Golem: fair, secure and that can reflect the behavior of real-world economies.
But the activities of the RnD team do not limit to economics: our ultimate goal is to strive for opportunities and possibilities for Golem to implement new technologies like Swarm, IPFS, Plasma, trusted environments, etc.
Unlimited
During our analyses of demands of computing centers, rendering farms and owners of existing hardware infrastructure, we noticed that for these users installing Golem within their current architecture is rather inconvenient. Nowadays, each Golem node acts as an independent market participant. Hence, the lack of trust between nodes requires to use less-efficient algorithms and protocols.
Unlimited is aimed to supplement Golem’s functionality with the wide range of features that trusted environments offer. The product introduces the possibility of joining nodes into trusted groups and run any calculations within these groups, according to each group policy.
CGI Team
The CGI team focuses mainly on four topics. The first of them is the verification of rendering results — currently, we have already implemented a local verifier, that operates by comparing locally rendered image fragments with the received bitmap.
We want to improve this method by involving the nodes running SGX into the process of computing the reference images, so the requestor will not need to do the process in their machine.
Another mechanism of verification will be approached by this team. The providers will render partially overlapping image areas, and then the overlapping fragments will be compared. This method — we call it redundant verification — is going to take into account the providers’ reputation.
The team will also approach the topic of compositing. We will provide mechanisms for handling postprocessing in Blender. Currently, some of these software effects don’t work properly on Golem, so fixes are being analyzed and will be implemented. On this front we are working on a Blender plugin — to ease the process of dispatching tasks to the network, which hopefully will grow the userbase.
Lastly, the final and more ambitious topic is new use-case implementation — video transcoding and commercial renderers integration. We are hoping this will push the CGI marketplace forward.
A Glimpse into the future of Golem
CGI and its extensions have been always considered as a starting point, the tip of the Golem use-cases iceberg, metaphorically, not the ultimate goal for our project. This is why we have created an internal process and a dedicated team whose core responsibility is to bring novel and promising use cases on board and make sure the technology behind Golem can accommodate their requirements.
Golem’s ‘Use Case Pipeline’ consists of internal analysis and development as well as cooperation with several business partners. We are open to integrating various potential use cases, which fit well into Golem’s architecture.
Such use-cases originate from a variety of fields such as machine learning, computational chemistry, financial market analytics and many other possibilities, which dramatically widen the reach spectrum of the software and the impact it can have.
Our Use Case Pipeline welcomes both owners of well-established software (including proprietary solutions) as well as projects which not yet have reached the production phase.
Individual empowerment as a core value for Golem and the way forward
Our team sticks together yet our mindset has changed, for the best.
Recently our internal organization has undergone a drastic and agile transformation. We evolved from a traditionally managed approach to one set to focus on the empowerment of the individuals that comprise the Golem team. For this, the dev team got divided into 6 smaller teams. In addition, we moved to our new office and we will endeavour to deliver more and faster, yet maintaining our premises of innovation and openness to changes. This is directly tied to our goal to recruit vastly talented engineers to tackle problems like untrusted providers, blockchain scalability related issues, machine learning or distributed reputation. You can check our open positions, and apply!
Obviously, this new mid-term goals Kanban cannot gauge all the planning around Golem, our commitment towards mass adoption of the decentralized ecosystem, and to progress.
On the side of our technical growth, but strictly tied to it, we continue to work in other areas such as business development, communication, marketing and more. Additionally, we make a pledge to deliver the best and most novel technology out there, and for this purpose, we need to provide education to the users, such as in the latest “Golem’s Guide to Trusted Computations” blog post series, outreach through conferences, and above all, we continue to grow our commitment to Open Source Software development by making our tools and technology available to a wider audience.
As always, thanks to our community for being always there. The team is stronger than ever, and its morale is always high so we will continue progressing at every step of the way.
Watch yesterday’s livestream here!
Have any further questions? Our #roadmapquestions Rocket channel stays active!
Follow us on Medium and Twitter for the next updates