SEI DB: Why Database Optimization is essential for scalability

Author: Steve Source: 4pillars Translation: Shan Ouba, Bit Chain Vision Realm

Point

• “Parallel execution” is one of the keywords in the blockchain industry in the past year.However, in order to truly implement parallel implementation, it also needs innovation in various fields. The most important of which is the improvement of the blockchain database.

• One of EVM parallel processing SEI has realized this necessity and has been considering database optimization since last year.The results of these efforts are SEI DB.

• SEI DB transforms the traditional single database structure into a modular structure divided into two layers.It eliminates unnecessary metadata and optimizes status access, thereby eliminating the low efficiency of the database level and improving the overall performance of the blockchain.The SEI method is not only a good example for blockchain pursuing parallelism, but also for builders who aim to improve the overall blockchain efficiency.

In 2023 and 2024, there are many keywords in the blockchain market cycle. Meme coins, Farcaster, RESTAKE, etc. are among them.However, I found that the most interesting keywords in technology are “daily implementation”.Although the market is familiar with EVM parallel processing, I think that through parallel transactions, it has fundamentally improved blockchain performance than EVM itself.

Speaking of “trading parallel execution”, you may think of various chains, but the first thing I think of is the SEI blockchain.They are not the first person to introduce the concept of parallel processing in parallel, but they play an important role in popularizing this keyword in the market.As of writing this article, SEI Network has become the first Layer 1 blockchain to be processed in parallel processing.This is because they have passed a governance proposal that supports EVM parallel processing.

The passage of the SEI V2 governance proposal is of great significance, because it shows that the parallel processing of the parallel processing that it has been considered difficult to implement has reached a practical stage.Why is it so difficult to apply for parallel transactions?My survey revealed several reasons.First of all, conflicts are likely to occur between transactions that affect the same state (such as changing the same account balance).When determining the transaction order, the complexity will increase.The most important thing is that even if the affairs are parallelized in the execution layer, it is difficult to achieve significant improvement of scalability without database -level optimization.These issues make the implementation of parallel processing in parallel.

SEI’s co -founder and CTO Jayendra has been pointed out through various media channels (database -level optimization).If EVM parallel processing or general transaction parallel processing is only regarded as the optimization of the “execution” level of the blockchain, significant scalability improvement cannot be achieved.Therefore, to discuss the performance of parallel processing, it is necessary to solve how to achieve database -level optimization.

Today I want to talk about how the SEI blockchain optimizes its database.Please note that database optimization is not just a problem of supporting parallel processing; for the blockchain that supports parallel processing, database optimization is also a problem.This is a challenge facing all high -performance blockchain that needs to handle a large number of transactions.Through this article, I hope readers can learn more deeply for SEI V2, and I also hope that those who design high -performance blockchain can get valuable insights to design databases in high -performance blockchain.

1. Question of blockchain storage: state expansion

1.1 What is the state in the blockchain?

Before discussing the storage problem, we need to define the meaning of the state.What is the state in the background of the blockchain?Status refers to the information of all accounts in the blockchain, including the detailed information of the account itself, account balance and contract code.Therefore, when trading on the blockchain, it inevitably affects a specific state.For example, if A is transferred to B to B, the balances of A and B need to be updated.This is the meaning of the country’s change.What impact does the state change?Generally, we do not think that the state is only increased by changes, but even the transaction that only changes the state will leave trading records in the history of the blockchain (this type of data is called a historical state).Therefore, it can be said that even the transaction with state changes will slightly increase the state size.In other words, all the transactions on all chains help the country’s growth.

1.2 The status growth problem in the fast blockchain

As I explained before, the chain transaction helps the state growth.For a fast blockchain like SEI, it handles more transactions within a given time. Compared with other blockchain, the state growth rate will be faster.If we further add support for the parallel implementation of transactions, the status will grow faster, which will lead to several problems:

1. Increase the operating cost of node: The entire node must store the entire blockchain status, which increases the storage cost and makes it difficult to operate the node on the blockchain.This may lead to centralization because the entry threshold for running a full node becomes higher.

2. Blockchain performance decreases: a large state means that nodes need more time to process and verify transactions.Whenever a transaction that changes the state of the blockchain, nodes need to read and update related status values.With the growth of the state, more data needs to be accessed, and more state value needs to be changed.This will eventually lead to slower blockchain performance.

3. Node synchronization problem: Blockchain is fundamentally involved in multiple nodes shared ledger and continuously synchronized and effective ledger.If the state becomes too large, the addition of new nodes will become very difficult.The new node must download the entire ledger of the chain to participate in the main network.The chain takes the “snapshot” of the entire record at a specific point. The new nodes use these snapshots to synchronize.However, if the chain is too large, it takes a long time to take a snapshot, and during this period, the blockchain continues to add new transactions and data.This difference may make the synchronization of new nodes difficult.Simultaneous and backward nodes will also face a lot of time and costs to catch up, which makes performance problems and make new nodes more difficult to join, which may lead to centralized problems.

The problem that becomes too large in the blockchain is called state expansion.If the transaction is implemented parallel without database improvement, the status will be further expanded, leading to various problems.These issues eventually hinder the realization of the benefits brought by the parallel implementation of affairs.SEI has realized this problem from the beginning, and the result of this understanding is SEI DB.So, what does SEI DB focus on design and how much does the database improve?

2. Introduce SEI DB, the fastest blockchain and fastest database

The SEI V1 uses ordinary database structures based on non-changing Adelson-Velsky and LANDIS (IAVL) tree structures. COSMOS SDK uses this structure to store status.In Ethereum, a similar concept is Merkle Patricia Trie (MPT).However, this structure is proven to be inefficient in several aspects: 1) it needs to store a large amount of metadata on each node.) It usually causes more metadata.The data exceeds expectations.In order to solve these inefficient problems, SEI launched SEI DB, which is a modular database architecture that divides the storage into two layers.

Why divide the database into two layers?Because the country itself is divided into historical state and active state.In order to better understand the SEI DB, it is necessary to define the two types of these two types:

1. Historical state

Historical state refers to all the state recorded before the blockchain’s latest block height.In other words, it contains all the past records of the blockchain, excluding the currently processed blocks.For example, all the records of users in the past are historically state.

1. Activity status

Activity status involves information related to the latest block height.Simply put, it includes the latest information recorded on the blockchain, such as the current balance of the user.

Even from these definitions, it is clear that the historical state and the status of the activity contain obvious different information, and the historical state is much heavier than the activity status.SEI aims to optimize the database by handling these two types of states in different places.

SEI DB is divided into 1) status commitment (SC layer) and 2) Status storage layer (SS layer).Let’s study the characters of these layers and how they interact.

2.1 Status promise layer (SC layer)

Status promise layer to manage the status of the SEI blockchain.The most critical component of the SC layer is memory mapping IAVL tree (Memiavl), which is a modified version of the IAVL tree used in COSMOS SDK.The modification of low efficiency mentioned earlier and optimized the status access.But why is Memiavl so effective for status access?To understand this, we need to study the concept of memory mapping used by SEI.

Generally, when processing files, use file descriptive or file structure pointers to access them, which requires input and output operations through a buffer.Memory mapping (mmap ()) solves this problem by mapping the file to the virtual address space of the process, allowing reading or writing data without using read or writing functions.

According to SEI, Memiavl can achieve status access within hundreds of nail seconds. Compared with the traditional tree structure, the writing speed is 10 to 287 times faster and the read speed is 10 times faster.

>

(The above figure focuses on writing (submitting) instead of status reading. This result shows that the combination of SEIDB and asynchronous submission has achieved significant performance improvements.)

In order to facilitate understanding, let’s outline the entire life cycle of transactions recorded on the blockchain using Memiavl:

1. When transaction occurs, read the status from Memiavl, and transaction execution will lead to status changes (also known as changes)

2. The change set is first applied to the MEMIAVL tree, and then you can re -calculate the new Hash.

3. The newly calculated status root value is included in the block head, and the label transaction has been successfully recorded on the blockchain.

4. In different Goroutine, these changes are asynchronous asynchronous to a WAL file and can be used to recover (if you need to restore a node, you can use the recent snapshot and information stored in WAL for recovery).).).

These changes have fundamentally reduced the use of CPUs and memory, enabling SEI to build abnormally fast blockchain without the need for high hardware specifications.

2.2 Status storage layer (SS layer)

The most critical activity status of the state submitting layer management, and all the records before the state storage layer processing activity status, also known as the historical state.The status storage layer of the SEI V2 consists of three key components. We will study them in detail:

2.2.1 Original key value storage

Anyone who is familiar with the blockchain will encounter the concept of key value pair.The data storage structure uses key as the unique identifier and the use value is used as associated data.For example, in the context of the blockchain, the account balance or contract status is represented by the key, and the corresponding data (such as the number of token in the account) is the value.

Although key value storage structures are common in other blockchains and databases, the SEI is optimized by minimizing meta -data storage, thereby reducing the amount of storage data.In addition, because the key is directly mapped to the value, there is no additional abstraction layer, so the data access speed is faster, thereby improving the overall efficiency of the blockchain.

2.2.2 Flexible database back end

The efficiency of the database structure is equally important as its support for various storage backends.It is required that the use of a single storage back end to be required to use a single storage back end may be restricted, because this will prevent them from optimizing the back end to meet their specific needs.SEI V2 supports Pebbledb, Rocksdb and SQLITE, allowing nodes to select the database that is most suitable for its needs.The characteristics of these three databases are compared below:

>

The characteristics of the database supported by SEI V2 are consistent with the performance of SEI.Each of these databases is optimized, which can effectively process large -scale data and minimize the large -scale writing and release (that is, reduce the frequency of data writing to the disk).

SEI said that Pebbledb showed the best performance in the supported database.However, it is worth noting that these databases have their own advantages and disadvantages.For detailed advantages and disadvantages, you can refer to the comparison chart provided by the SEI team.

2.2.3 asynchronous pruning

Finally, the component to be discussed is asynchronous trimming.In the context of the blockchain, trimming refers to the process of deleting unnecessary or outdated data from the blockchain.Traditionally, the trimming operation will have a negative impact on network performance.However, the SEI asynchronous execution of pruning operations means that these tasks are implemented in the background, which will not affect the main blockchain operations.This method allows SEI to optimize historical status data and reduce the storage needs of nodes without affecting the performance of the blockchain.

In summary, the innovative database management method of SEI V2, including the original key value storage, flexible database back -end support and asynchronous pruning to ensure effective processing activities and historical status data, thereby improving the overall performance of the blockchain and scalable expansion of the blockchainsex.

3. The implementation results of SEI DB

>

We have now explored the two layers of SEIDB (status submission layer and status storage layer) and studied the characters and functions of each layer.Through this explanation, we learned that SEIDB theoretically enhanced the performance of the SEI blockchain through databases and optimized it.However, the most important thing is the actual result.When the SEI team implements SEIDB in the test network environment, the following data is obtained:

1. Reduce activity status

The size of the activity status stored in memory was measured, and the results showed that the size of the activity was reduced by 60%.

2. The growth rate of historical data decreases

Evaluate the growth rate of the historical state, and it is found that it is more than 90%slower than the previous database.

3. Synchronization time is reduced

The time required to measure the synchronization state of the node showed that the speed increased by about 1,200%.

4. Lottery time decrease

The time required from the synchronous block of the snapshot point to the latest block height, the display speed is doubled than the previous database.

5. A decrease in block submission time

The time required to submit the block to the blockchain was measured, and the result showed that the speed was increased by 287 times compared with the previous database.

6.TPS (Number of transactions per second)

The time required for the processing office was measured, and the results showed that the speed was increased by more than doubled compared with the previous database.

Based on these indicators, the implementation of SEI V2 through SEIDB will show significant performance improvement.Although it is covered by the main narrative of EVM compatibility, the long -term improvement of SEI is likely to change in database levels.

4. Looking forward to the future: beyond the narrative and make actual contributions

The SEI V2 era has arrived.In the crucial market, SEI V2 is generally reminiscent of “EVM parallel processing”.However, if you carefully study the changes brought about by the V2 upgrade, you will find that technology -intensive transformation far exceeds the scope of EVM support and parallel processing improvement.Although the performance indicators I mentioned earlier are only the test results before the V2 main network released, the actual impact still needs to be observed.Nevertheless, these efforts are worthy of continuous attention, because the actual performance of SEI V2 can inspire other Layer 1 blockchain to test and enhance its database, thereby making a wider goal of “blockchain performance improvement”.Major contribution.

From the beginning, SEI has been pursuing a single goal of becoming a “fast blockchain” and conducted extensive research to achieve this goal.As a researcher, I praise them unremitting dedication to the implementation of fast blockchain.In addition, I hope that their research can continue to develop and bring better database innovation. These innovations may come from the SEI team itself.These efforts will enable us to build an excellent blockchain, and ultimately make blockchain technology more likely to be used by more extensive audiences.