Fundament of the Week: What is sharding?

What is sharding?

Sharding is a form of database partitioning, also known as horizontal partitioning. This process involves splitting a very large database into smaller, more manageable segments with the idea of improving performance and reducing request response time.

Sharding is not a new concept and has existed in traditional centralized database management since at least the 1990s. This term was popularized by one of the first mass multiplayer online role-playing games (MMORPG) - Ultima Online. In that, the developers divided the players into different servers (worlds) to deal with the traffic better.

In business, a common example of sharding is the division of customer databases according to geographical location . Customers that are sharing similar geographical locations are grouped and located on the same servers.

What is blockchain sharding?

It is basically the same process. Blockchain is a database whose nodes represent individual data servers. If we use sharding on a blockchain, it would mean its division into individual segments (shards) . Each shard would have a unique set of smart contracts and account balances.

The nodes would then be assigned to individual shards to authenticate transactions and operations, instead of every single node being responsible for verifying each transaction across the network.

The point is that splitting the blockchain into more manageable segments should increase transaction throughput, and thus overcome scalability issues that most blockchains are facing today.

How does sharding work?

For this explanation, we will use the Ethereum blockchain as an example. It is made up of thousands of computers called nodes. Each of the nodes lends a certain amount of hashing energy to the network. It is this hashing energy that enables the operation of the Ethereum Virtual Machine (EVM) - i.e. the execution of intelligent contracts and the launch of decentralized applications (DApps).

Currently, Ethereum operates on a sequential basis, with each of these nodes having to count each individual operation and process each transaction. Therefore, a transaction that goes through this authentication process can take a long time. At present, Ethereum is around 3,000 transactions per second (TPS). For comparison - VISA can average up to 24,000 TPS. Adding computers to the network does not necessarily improve efficiency, as the entire ledger is stored on every single node, only extending the authentication chain.

The idea of moving from a linear execution model , in which each node must count each operation, to a parallel execution model , in which nodes are assigned to process only some calculations, is linked to sharding. This will allow multiple parallel transaction processing simultaneously. The blockchain will be divided into separate shards. Nodes will only need to run the part of the ledger assigned to them in order to execute processes and verify transactions, instead of running the entire ledger.

What problem can be solved by sharding?

Sharding is a potential solution to persistent blockchain scalability issues. As mentioned above, one of the biggest problems that are blockchain networks nowadays facing is the issue of scalability. The more popular the blockchain network, the more users initiate transactions, decentralized applications, and other processes on the network.

Increased transaction activity puts increased demands on nodes that work to verify transactions, and there is a real risk that these blockchains may become clogged (as we saw on Ethereum, for example, during the CryptoKitties frenzy, when the game accounted for 11% of network transactions). If this happens, the speed of transactions will slow down significantly, which is not an ideal situation for the long-term sustainable adoption of a blockchain.

As we mentioned above, the blockchain is divided into smaller segments, with teams of nodes assigning responsibilities to individual segments. This means that each node will not have to maintain the entire ledger to perform each operation. Therefore, transaction confirmation may occur in a parallel rather than a linear manner, increasing the speed of the entire network. It provides a solution to the scalability issues of blockchain networks and therefore makes them more sustainable in the long run.

Who implements sharding?

Some blockchains have already implemented a protection mechanism, while others are still evolving. Zilliqa is the first public blockchain platform to implement sharding. In its test network, it managed to reach 2,828 TPS.

The Near blockchain ecosystem allows developers to easily create and deploy decentralized applications. Its sharding technology allows nodes to stay small enough to run on simple cloud-hosted cases — and mobile devices in the future. As in the case of Near, Ethereum provides a blockchain ecosystem for the implementation of DApps, based on smart contracts. Other blockchain projects that focus on sharding as a solution to scalability issues include Cardano, QuarkChain and Pchain .