Sui
Sui Lutris: The distributed system protocol
at the heart of Sui
May 31, 2023
Mysten Labs
4 min read
Update: August 18, 2023
After a few months of testing, we’ve updated the Sui Lutris whitepaper, validating that:
Sui can hit 140k – 150k operations per second using PTBs and 5K TPS, which indicates that Sui’s capacity benchmarked at the peak of Mainnet (~700 TPS) is well below what Sui can actually do.
Sui’s finality latency stays under ½ second even when a number of validators are down.
Within the Sui Lutris whitepaper, we’ve also included instructions on how external testers can replicate these numbers on Sui for their own validation tests. We’ve also included proofs of safety, and a detailed description of how Sui works.
Sui Mainnet was announced just a few weeks ago, and we’re already seeing exciting applications on-chain, such as games and NFTs. Today, we’re releasing a technical report on “Sui Lutris,” the distributed system protocol that powers Sui, allowing it to achieve very low latency alongside high-throughput and long-term stability.
Since the inception of Bitcoin a decade ago, blockchain technology has come a long way. With the emergence of new use cases, such as gaming and non-fungible tokens (NFTs), the blockchain community is constantly exploring ways to improve the efficiency of the technology, particularly in handling high loads and providing real-time latencies.
We believe L1 blockchains face two major challenges. They need to achieve high throughput while maintaining low latency, and ensure the stability of the consensus protocol over long periods of time. These challenges should be solved while preserving decentralization through dynamic participation of validators and reconfiguration of the validator set.
One promising approach to achieve high-throughput is the use of DAG-based consensus protocols, such as Narwhal / Bullshark, which Sui uses. These consensus protocols enable blockchains to order a large number of transactions simultaneously, making them ideal for use cases such as gaming and NFTs. However, DAG-based protocols induce latencies of a few seconds, which is quite high for common transfers or game actions.
Consensus-less protocols on the other hand, such as our past research prototype FastPay, have shown great promise in reducing latency and scaling. These protocols eliminate the need for consensus, allowing quick processing of transactions without the need for total ordering of independent transactions that are handled in parallel. However, they are limited to a restricted class of simple blockchain operations. This limits the smart contracts they can express, and it can be challenging to reconfigure the dynamically changing validator set.
Despite their potential, neither of these approaches are currently used in production-grade blockchains. They are currently only presented at academic conferences, and are not widely used by the blockchain community. Sui Lutris, the protocol underpinning Sui, combines DAG-based consensus with consensus-less approaches to achieve the best of both worlds: sub-second latencies and sustained throughput of thousands of transactions per second. Sui accomplishes both of these and maintains the ability to express complex contracts on shared objects, produce checkpoints, and reconfigure the validator set across epochs.
Combining Consensus-Based and Consensus-Less Approaches
Sui Lutris takes a unique approach by combining the two approaches mentioned above. To ensure the safety of operations on assets owned by a single owner (owned objects), the system employs a consistent broadcast protocol between validators. This results in lower latency than consensus. Sui Lutris relies only on consensus to process complex smart contracts that operate on shared objects, that is objects that any user can mutate. Sui Lutris also supports network maintenance operations, such as defining checkpoints and reconfiguring validators. This novel strategy offers the best of both worlds when processing transactions in a replicated Byzantine environment.
A high-level description and the transaction lifecycle in Sui Lutris.
A high-level description and the transaction lifecycle in Sui Lutris.
A user with a private key creates and signs a user transaction to either mutate objects they own, or a mix of objects they own and shared objects (➊). The transaction is sent to each Sui Lutris validator (often through a full node), which performs a series of validity and safety checks, signs it, and returns the signed transaction to the client (➋). The client collects the responses from a supermajority of validators to form a transaction certificate (➌). At this point the transaction can be considered irreversible (finality is reached).
Once the certificate is assembled, it is sent back to all validators, who check its validity and acknowledge its receipt to the client. If the transaction involves exclusively owned objects the transaction certificate can be processed immediately and executed without waiting for the consensus engine (direct fast path). All certificates are forwarded to our DAG-based consensus protocol (also operated by the Sui validators) (➍). Consensus eventually outputs a total order of certificates; the validators check and execute those containing shared objects (➎). Clients can collect a supermajority of validator responses, assemble them into an effect certificate, and use it as proof of the settlement of the transaction (➏). Subsequently, checkpoints are formed for every consensus commit (➐), which are also used to drive the reconfiguration protocol.
Besides the illustrated main transaction flow, Sui Lutris provides a number of facilities to support a production grade blockchain:
A checkpointing protocol is implemented after finality is reached which produces a causal history of all transactions in the system. This is used for complete audits, as well as to keep Full nodes and lagging validators synchronized in an efficient manner.
Sui Lutris supports reconfiguration at the end of each epoch, when the set of validators and their voting power might change. To ensure that all final transactions are included in an epoch, each epoch needs to close carefully and confirm that finality is safe.
In previous consensus-less protocols, assets are locked forever when a buggy client double-spends an asset. Sui Lutris safely “unlocks” mistakenly locked assets at the end of the epoch, minimizing the damage of bugs.
Sui Lutris underpins Sui, a blockchain managing significant amounts of value for its users. The complete technical report provides additional detail on how the safety and liveness protocols operate, as well as proofs of security for them in the standard distributed systems model with Byzantine participants in partial synchrony.
See the complete Sui Lutris paper for more details.
Update: August 18, 2023
After a few months of testing, we’ve updated the Sui Lutris whitepaper, validating that:
Sui can hit 140k – 150k operations per second using PTBs and 5K TPS, which indicates that Sui’s capacity benchmarked at the peak of Mainnet (~700 TPS) is well below what Sui can actually do.
Sui’s finality latency stays under ½ second even when a number of validators are down.
Within the Sui Lutris whitepaper, we’ve also included instructions on how external testers can replicate these numbers on Sui for their own validation tests. We’ve also included proofs of safety, and a detailed description of how Sui works.
Sui Mainnet was announced just a few weeks ago, and we’re already seeing exciting applications on-chain, such as games and NFTs. Today, we’re releasing a technical report on “Sui Lutris,” the distributed system protocol that powers Sui, allowing it to achieve very low latency alongside high-throughput and long-term stability.
Since the inception of Bitcoin a decade ago, blockchain technology has come a long way. With the emergence of new use cases, such as gaming and non-fungible tokens (NFTs), the blockchain community is constantly exploring ways to improve the efficiency of the technology, particularly in handling high loads and providing real-time latencies.
We believe L1 blockchains face two major challenges. They need to achieve high throughput while maintaining low latency, and ensure the stability of the consensus protocol over long periods of time. These challenges should be solved while preserving decentralization through dynamic participation of validators and reconfiguration of the validator set.
One promising approach to achieve high-throughput is the use of DAG-based consensus protocols, such as Narwhal / Bullshark, which Sui uses. These consensus protocols enable blockchains to order a large number of transactions simultaneously, making them ideal for use cases such as gaming and NFTs. However, DAG-based protocols induce latencies of a few seconds, which is quite high for common transfers or game actions.
Consensus-less protocols on the other hand, such as our past research prototype FastPay, have shown great promise in reducing latency and scaling. These protocols eliminate the need for consensus, allowing quick processing of transactions without the need for total ordering of independent transactions that are handled in parallel. However, they are limited to a restricted class of simple blockchain operations. This limits the smart contracts they can express, and it can be challenging to reconfigure the dynamically changing validator set.
Despite their potential, neither of these approaches are currently used in production-grade blockchains. They are currently only presented at academic conferences, and are not widely used by the blockchain community. Sui Lutris, the protocol underpinning Sui, combines DAG-based consensus with consensus-less approaches to achieve the best of both worlds: sub-second latencies and sustained throughput of thousands of transactions per second. Sui accomplishes both of these and maintains the ability to express complex contracts on shared objects, produce checkpoints, and reconfigure the validator set across epochs.
Combining Consensus-Based and Consensus-Less Approaches
Sui Lutris takes a unique approach by combining the two approaches mentioned above. To ensure the safety of operations on assets owned by a single owner (owned objects), the system employs a consistent broadcast protocol between validators. This results in lower latency than consensus. Sui Lutris relies only on consensus to process complex smart contracts that operate on shared objects, that is objects that any user can mutate. Sui Lutris also supports network maintenance operations, such as defining checkpoints and reconfiguring validators. This novel strategy offers the best of both worlds when processing transactions in a replicated Byzantine environment.
A high-level description and the transaction lifecycle in Sui Lutris.
A high-level description and the transaction lifecycle in Sui Lutris.
A user with a private key creates and signs a user transaction to either mutate objects they own, or a mix of objects they own and shared objects (➊). The transaction is sent to each Sui Lutris validator (often through a full node), which performs a series of validity and safety checks, signs it, and returns the signed transaction to the client (➋). The client collects the responses from a supermajority of validators to form a transaction certificate (➌). At this point the transaction can be considered irreversible (finality is reached).
Once the certificate is assembled, it is sent back to all validators, who check its validity and acknowledge its receipt to the client. If the transaction involves exclusively owned objects the transaction certificate can be processed immediately and executed without waiting for the consensus engine (direct fast path). All certificates are forwarded to our DAG-based consensus protocol (also operated by the Sui validators) (➍). Consensus eventually outputs a total order of certificates; the validators check and execute those containing shared objects (➎). Clients can collect a supermajority of validator responses, assemble them into an effect certificate, and use it as proof of the settlement of the transaction (➏). Subsequently, checkpoints are formed for every consensus commit (➐), which are also used to drive the reconfiguration protocol.
Besides the illustrated main transaction flow, Sui Lutris provides a number of facilities to support a production grade blockchain:
A checkpointing protocol is implemented after finality is reached which produces a causal history of all transactions in the system. This is used for complete audits, as well as to keep Full nodes and lagging validators synchronized in an efficient manner.
Sui Lutris supports reconfiguration at the end of each epoch, when the set of validators and their voting power might change. To ensure that all final transactions are included in an epoch, each epoch needs to close carefully and confirm that finality is safe.
In previous consensus-less protocols, assets are locked forever when a buggy client double-spends an asset. Sui Lutris safely “unlocks” mistakenly locked assets at the end of the epoch, minimizing the damage of bugs.
Sui Lutris underpins Sui, a blockchain managing significant amounts of value for its users. The complete technical report provides additional detail on how the safety and liveness protocols operate, as well as proofs of security for them in the standard distributed systems model with Byzantine participants in partial synchrony.
See the complete Sui Lutris paper for more details.