How Shardeum Achieves Low Latency Enabling Dapps & Users With a Smooth UX

Introduction

Latency, in general, refers to the time it takes for data to travel from its source to its destination. Network latency can be described as the delay or pause between the moment data is transmitted from one point and when it is received at its intended destination. Factors such as long distances, multiple device hops, and inadequate infrastructure can contribute to higher-than-desired latency.

What Is Low Latency and Why Is It Important?

Low latency refers to a system's ability to minimize the delay between data transmission and reception as much as possible, which can have a significant impact on user experience. Reducing latency is a fundamental aspect of contemporary technology and is being extensively optimized for various industries, including blockchain networks. Whether it's gaming, financial trading, or telecommunications, low latency is the essential element that real-time applications and systems require for immediate response times.

For example, low latency is vital for high-frequency and algorithmic trading in the financial sector, where even milliseconds can have a substantial impact on a trader's position. Online gaming is another domain where low latency is of paramount importance, enabling players to respond swiftly and efficiently in fast-paced games. This guide will delve into the advantages of low latency for smart contract platforms and how Shardeum, as an L1 network, will uphold remarkably low latency levels—a historical challenge for the blockchain industry.

Latency, Finality & Scalability in Blockchain Networks

Latency, within the context of blockchains, denotes the complete time taken from submitting a valid transaction to a blockchain network to the network processing and confirming the said transaction. It's important to note that latency and finality are closely related in blockchains, although they are not identical. Finality, on the other hand, pertains to the irreversibility of a confirmed transaction once it's added to the blockchain network. Thus, finality time refers to the overall time it takes from submitting a valid transaction to a blockchain network until the network finalizes the transaction. At this juncture, the transaction is considered final and cannot undergo any further alterations. Consequently, achieving very low latency and finality is of utmost importance, not only for scaling up the network but also for processing a remarkably high number of transactions per second (TPS).

Similarly, latency and scalability are interconnected, yet they are distinct concepts. Low latency is centered on reducing the time required to process individual transactions, whereas high throughput or scalability (typically measured in TPS) underscores the blockchain network's capacity to manage a substantial influx of transactions concurrently.

Latency Time – Web2 vs Web3 Networks

Centralized networks, which encompass private blockchains, typically exhibit shorter latency times when compared to decentralized blockchain networks. The primary reason for this variance in latency lies in the architectural and processing mechanisms employed by these two distinct systems. Centralized servers are under the control of a single entity or organization, enabling them to swiftly process and validate transactions, as there is no necessity for multiple parties to reach consensus on each transaction's validity.

Latency in centralized systems is primarily contingent on factors such as the speed and capacity of the server's hardware, the efficiency of the software, and network conditions. In addition to employing techniques like load balancing and network maintenance, these entities address lag times by vertically scaling, which involves enhancing the processing power of their system hardware.

How Higher Latency Affects the Blockchain Industry Today?

1. Addressing the Scalability Trilemma Poses Challenges

Decentralised networks distribute the workload across numerous individual nodes/servers globally and subsequently reach consensus on transaction validity before recording transactions in the network's distributed ledger or blockchain. The choice of consensus algorithm and self-imposed scaling limits are pivotal for public networks because they are fundamental to the blockchain's core Unique Selling Points (USPs) – security and decentralisation.

Blockchain platforms adjust their block sizes based on network traffic and devise an appropriate consensus algorithm accordingly. Larger blocks result in a higher Transactions Per Second (TPS) but carry the risk of centralisation and susceptibility to attacks. Bitcoin, with its fixed smaller blocks, prioritises security over smart contracts. In contrast, Ethereum expanded the blockchain's potential by enabling smart contracts and a broader range of commercial use cases. Networks of this nature often prefer a diversified or hybrid block size, allowing them to process a slightly higher number of transactions per second while still imposing size limits to prevent centralisation and potential attack vectors.

As a drawback, this naturally brings about delays and reduced speed, which have made decentralised blockchains less suited for real-time applications and utilities such as online gaming, supply chains, telecommunications, and healthcare, up to this point. This is why there is a competition among the latest networks to address the blockchain trilemma – wherein they aim to consistently uphold security, decentralisation, and scalability.

2. Latency & Finality are Inversely Proportional to Throughput

An important aspect to consider is the inverse relationship between latency and throughput. This means that as latency and finality time increase, throughput decreases, limiting a network's capacity to process a high number of transactions per second (TPS). This has been the case with traditional systems and is indeed applicable to blockchain-based networks, at least at a foundational level.

However, blockchain network latency involves more complex components such as smart contracts and consensus mechanisms, aiming to provide real-world solutions without the need for intermediaries. It relies heavily on a high degree of automation through smart contract coding. That being said, low latency plays a more significant and active role than, for instance, bandwidth requirements in determining the scalability and throughput of a blockchain network. In fact, a blockchain network can perform well with lower data bandwidth, provided it incorporates an optimal architecture designed to minimize network finality.

3. High and Unpredictable Transaction Expenses

The end result of vertical scalability, low TPS, and the inability to resolve the scalability trilemma is elevated transaction expenses and gas fees, particularly when network traffic surges. Higher latency compounds the issue, making it more costly and slower for users and dapp developers to engage with a smart contract platform due to increased computation and communication expenses.

Despite the concept of decentralisation, many of the tools and services facilitating decentralised ecosystems today still operate in a predominantly centralised manner. To avert network congestion, outages, and subpar user experiences, public blockchains often resort to vertical scaling by enhancing the computing power of their nodes. Nevertheless, this approach can render it financially unfeasible for everyday users to maintain nodes on these networks, ultimately driving up transaction costs for both users and dapp developers.

4. MEV and Front-Running Threats

Public blockchains transparently record transactions. In addition to self-imposed scaling limits, validators have found themselves motivated to prioritize high-fee transactions as an unintentional outcome, aiming to mitigate network congestion and discourage excessive traffic. However, validators seldom deviate from network protocols to the extent of manipulating the entire market, primarily due to the substantial costs, such as slashing, associated with such actions and the resilient consensus mechanism itself.

The situation takes a troubling turn when malicious actors engage in similar front-running practices. They gain access to transactions, particularly larger ones, before confirmation and then position their own transactions ahead of them to profit from price discrepancies. In doing so, they not only aim to gain an unfair advantage over other users but also effectively manipulate the broader market, resulting in losses for the general public. Research conducted by Flashbots and the University of California suggests that, in 2023 alone, the cost borne by average Ethereum users due to front-running by malicious actors exceeded $2 billion. The adverse impact on the crypto market could be significantly larger.

Low latency and, consequently, immediate finality assume greater significance in this context as they make it more challenging for malicious individuals to access transactions before their confirmation on blockchains. Low latency blockchains can process transactions more expeditiously, leaving attackers with less time to observe and exploit transactions. When the confirmation time for transactions varies according to network demand, maximizing the benefits of even a timestamp-based autonomous ordering protocol becomes unfeasible.

5. Bad User Experience (UX)

As you can imagine, attaining widespread adoption of blockchains and the dapps constructed on them relies heavily on providing a positive user experience (UX) and keeping transaction costs to a minimum. We've discussed how the current ecosystem falls short in these aspects due to the design and construction of blockchains. Centralised systems, as mentioned earlier, recognised this issue long ago and have subsequently introduced various techniques to enhance latency, including Content Delivery Networks (CDNs), edge computing, the continual deployment of powerful data centres worldwide, and load balancing.

The goal is straightforward – to bring computing, data, and storage closer to end users, ensuring that response and load times are virtually instantaneous in a world that increasingly demands real-time data processing. This is the only way to meet the requirements of today's industries, spanning from AI and autonomous vehicles to algorithmic trading.

Elevated latency and delayed finality, for example, render decentralized cryptocurrency exchanges (DEXs) that operate via smart contracts less efficient compared to centralized exchanges (CEXs) using centralized protocols. As previously mentioned, heightened latency means that transactions may take a longer time to process and confirm, while slow or uncertain finality implies that confirmed transactions could potentially be reversed later on. This poses challenges for DEX traders who require swift trades with immediate settlement. Moreover, it gives rise to frequent slippages, resulting in unsatisfactory user experiences. Poor UX, in turn, contributes to systematically reduced liquidity in DEXs, which in turn fosters heightened volatility in cryptocurrency prices. Doesn't this scenario resemble the current state of the cryptocurrency economy?

6. Security Attacks

L1 blockchains prioritize high levels of security and decentralization, rendering them resilient against the sophisticated attacks prevalent in Web3 today. However, the same degree of security isn't always guaranteed for solutions and dapps constructed on L1 blockchains, even though they leverage the inherent security of these blockchains. This discrepancy arises because these solutions are crafted with end-users and their user experience (UX) in mind, often involving trade-offs that compromise decentralization and the robust security offered by L1 blockchains.

Security threats manifest in various forms, including phishing scams, malware attacks, and Denial of Service (DoS) attacks, among others. Of these, DoS and Distributed Denial of Service (DDoS) attacks have become particularly problematic even for newer L1 chains that emphasize scalability and low gas fees. These attacks involve inundating a network with numerous small transaction requests at a low cost, typically employing bots, thereby causing the network to slow down or become entirely inaccessible to legitimate users. Low latency can assist in mitigating this risk by making it more challenging for attackers to overwhelm the network, as they would need to send a larger volume of transactions.

Let's explore this analogy further. Imagine it as an attempt to flood a sink that drains quickly. If the sink (representing the network) efficiently removes water (symbolising requests) at a swift rate (low latency), you would need to pour a substantially larger amount of water simultaneously to cause it to overflow. Conversely, if the sink drains slowly (high latency), even a consistent trickle could eventually lead to overflow. In essence, low latency can considerably diminish the potential for various advanced attack methods, while high latency significantly amplifies vulnerability.

The Good News – Bigger Picture

The positive news for public blockchain networks and the broader Web3 ecosystem is that it inherently encourages the involvement of ordinary individuals, be it as validators, smart contract developers, or providers of storage resources. This sets it apart from its Web2 counterparts, where power primarily resides with companies that own data centres and cloud servers. More recent blockchain protocols are striving for elevated scalability right from the start, implementing optimal consensus mechanisms and other strategies such as auto-scaling to address the scalability trilemma.

How Does Shardeum Attain Low Latency?

Let's begin where we left off in the previous paragraph – by resolving the scalability trilemma.

Shardeum is an EVM-based layer 1 smart contract platform currently in its betanet stage. On Shardeum, consensus occurs at the transaction level rather than the block level, and transactions are processed concurrently using dynamic state sharding, resulting in very low latency, just a few seconds. That may sound a bit complex, so let's break it down in simpler terms. We'll provide a step-by-step explanation from the standpoint of blockchain technology so you can grasp the full picture.

1. Elevated Fairness & Transaction-Level Consensus

To commence, validators on Shardeum will assess and handle transactions in the order of first come first serve basis to virtually impossible for MEV and front-running transactions to thrive on the network. This elusive practice, also known as "high fairness," eliminates the initial bottleneck that impedes achieving low latency. Subsequently, validation, followed by consensus and processing, will be undertaken individually for each transaction. In traditional blockchain networks, a batch of validated transactions, determined by the block size, is amassed before being incorporated into blocks. Consensus and processing are executed for each block containing all the validated transactions. This naturally leads to heightened latency, as it necessitates time for all nodes, or even a subset of nodes, to process and ultimately confirm the blocks (a process that often unfolds sequentially in practice).

2. Immediate Finality

On Shardeum, each individual transaction that receives confirmation attains immediate finality, distinguishing it from other blockchains that offer either probabilistic or absolute finality. This marks a significant advancement in blockchain technology, as it delivers finality without the necessity to await confirmation from multiple blocks. Subsequently, these transactions are grouped together and transferred to the archive nodes within the network. This process enables validator nodes on the network to exclusively retain the current state of the accounts within the associated shards. We'll return to this point shortly. Staying on course, Omar Syed, co-founder of Shardeum, succinctly summarizes, "Once you've achieved consensus for the transactions (individually), the data structure (referring to blocks) you employ to store them becomes irrelevant. It holds significance for other networks because the data structure is intricately linked with the consensus.

3. Dynamic State Sharding & Concurrent Transaction Execution

Shardeum's dynamic state sharding will collaborate with its auto-scaling capability, enabling the network to autonomously adapt the quantity and dimensions of shards in response to the existing workload. Given that consensus occurs at the transaction level, a transaction that impacts multiple shards will undergo simultaneous processing rather than sequential processing. This not only diminishes the time required to process the transaction, even when it involves multiple shards, but also guarantees atomic processing with a network latency of just a few seconds.

4. Equal Distribution of Workload

In addition to facilitating the secure verification of transactions, Shardeum's leaderless consensus mechanism enhances network security and decentralization by periodically rotating validator nodes. Ultimately, Shardeum ensures an equitable distribution of workload among network resources while preserving operational efficiency as it expands and evolves within a sharded environment.

5. Linear Scalability & High Performance

Dynamic state sharding brings about the critical X factor for Shardeum, which is linear or horizontal scalability, as opposed to vertical scalability. Each node/shard introduced to the network from its pool of 'standby nodes' will proportionally enhance its TPS capacity, particularly during periods of high traffic. Shardeum's capacity to scale linearly and function effectively ensures that its network overheads, including latency, finality, and gas fees, remain highly foreseeable, even for intricate smart contract transactions.

Conclusion

In summary, Shardeum will enable ordinary individuals and computers to operate a node on the network, leading to consistently low fees, and providing end users with a user experience that mirrors what we are accustomed to in traditional Web2 networks. Shardeum's linear scalability will also apply to dapps developed on its network, ensuring they can swiftly and smoothly handle transactions, even as the network expands.

Frequently Asked Questions (FAQs)

1. Why is Low Latency Important?

The significance of low latency lies in its ability to diminish the lag in data transfer, leading to a quicker and more effective online encounter. This is particularly vital for online gaming, high-frequency trading, and healthcare applications, where receiving data in real-time can be a matter of life and death.

2. Is Low Latency Good?

Certainly. Low latency is deemed favorable because it leads to swifter and more effective data transfer, contributing to a seamless online experience. Nevertheless, the extent of this "favorability" varies depending on the particular use case, as different applications have distinct latency prerequisites.

3. Does Low Latency mean Faster?

Low latency implies quicker data transfer, leading to a swifter online experience. This is because low latency minimizes the delay in data transfer, facilitating more immediate interaction between applications and users.