jakiro
Source: Geek Web3
summary:
InSince the EIP-4844, the data throughput and storage pressure of the Ethereum network have increased, and the growing storage demand has brought huge challenges to Ethereum nodes.In order to reduce the storage pressure, some Ethereum client deletes the historical data stored in the local area, and the consistency of different nodes in storage behavior is gradually disintegrated.
InIn order to ensure that all Ethereum clients can reach an agreement on behavior,EIP-4444andEIP-4844 carried out historical data cutting behaviorstandardization, Will become the standard of Ethereum nodes in the future.
InTherefore, if historical data is to be replaced to restore the latest Layer1 or Layer2 status, it depends on the centralized, Ethereum protocol outside the Ethereum agreement, which prompts people to explore more decentralized, consistent storage solutions to Ethereum, and solve the solution to the Ethereum.plan
· Ethereum Portal NetworkIt is a lightweight, decentralized P2P network, which is suitable for all types of Ethereum data including historical data.It is designed for limited resources and provides Ethereum JSON-RPC services.The historical network and the letter chain network are almost ready.
· ETHSTORAGE is a incentive modular storage network for EIP-4844 BLOBS dataEssenceIn order to store BLOB, users can call the storage contract on L1, use ETH as the storage fee, and record the hash value of Blob on the chain.Over time, the storage cost will gradually be distributed to the storage service provider provided by the BLOB storage certificate under the chain.
InAt present, ETHSTORAGE test network is running online on the Ethereum Sepolia testing online, and many community participants have successfully proved their local storage conditions.Future plans include the development of a decentralized Ethereum state network, realizing the storage certificate for dynamic size data, and can directly access the Ethstorage network from the browser in a decentralized manner.
Thanks: Thanks to Piper Merriam from the Ethereum Foundation, Karthik Raju in Polychain, and Ethstorage’s qiang zhu’s feedback on this article.
background:
On October 22, 2023, Péter Szilágyi, the head of the famous Go-Ethereum (Geth) development, expressed his concerns about Ethereum data storage solutions on Twitter.He pointed out that although the Geth client retains all historical data, other types of Ethereum clients such as Nethermind and BESU can be configured to delete some Ethereum historical data (such as historical blocks).This will cause the behavior of some client nodes to be inconsistent with other clients, which is unfair to the Geth client operator.The above topic immediately triggered a fierce discussion about the storage scheme in the Ethereum roadmap.
Store challenge
Why is Nethermind and BESU allow client operators to cut local historical data?What is the problem reflected in this decision?
From our perspective, there are two main reasons:
-
The storage requirements of the Ethereum client become higher and higherEssence
-
Storage of Ethereum historical data does not have any incentives or punishment within an agreementEssence
The first reason originated from the rising storage needs of the Ethereum client.The following cake map shows the storage distribution of a new Geth node at 18,779,761 as of December 13, 2023.
As shown in the figure:
-
Total storage size: 925.39 GB
-
Historical data (block/transaction receipt): about 628.69 GB
-
Status data in Merkle Patricia Trie (MPT): about 269.74 GB
The second reason is that Ethereum node lacks incentives or punishments in the agreement that stores historical blocks.Although the agreement advocates all historical data of nodes, it has failed to provide any mechanism to encourage storage or punishment of violations.Nodes are willing to store and provide historical data extraction permissions, which are more for altruism, not because of incentives.
Of course, client operators can freely delete or modify all historical data without any punishment.In contrast, the Validator node must maintain and update the complete state locally to prevent the proposal/voting to support invalid blocks by SLASH.
Therefore, when storage costs become a major burden on nodes, it is not surprising that some node operators choose to delete historical data.In the absence of historical data, the node client can significantly reduce the storage cost and reduce the occupied storage space from about 1TB to about 300GB.
Figure: Nethermine’s configuration running node without historical blocks -currently saving about 460GB of storage costs
With the upcoming Ethereum data usability (DA) upgrade, storage challenges will increase.The road of comprehensive expansion of Ethereum DA began with EIP-4844, which is upgraded by Dencun. It introduces a fixed-size binary object (BLOB), and an independent cost model called BlobGasprice.Each blob is set to 128KB. After the EIP-4844 is implemented, each block contains up to 6 BLOB.In order to expand the data throughput, Ethereum plans to use the 1D Reed-Solomon deletion code, which initially allows 32 BLOB each block to reach the level of 256 BLOB per block during a complete expansion.
If Ethereum DA runs in full capacity (256 BLOB per block), the Ethereum DA network is expected to receive about 80 TB of DA data in one year, which is far exceeding the storage capacity of most nodes in the storage capacity of most nodes.Essence
Ethereum storage roadmap and its consequences
The Ethereum route push published by Vitalik mentioned that Purge mainly involves storage content.
The rising storage costs attracted the attention of Ethereum ecological researchers.In order to solve this problem and ensure the consistency of all clients,Researchers are developing some proposals to clearly delete historical data of the Ethereum client.The two main proposals are:
InEIP-4444: Restricting historical data in the execution client: This proposal allows clients to delete past blocks for more than one year.Assuming that the average block size is 100K, the upper limit of the historical block data is about 250 GB (100k * (3600 * 24 * 365) / 12, assuming the block time = 12 seconds).
InEIP-4844: Faber BLOB transaction: discard Blob data for more than 18 days.Compared with EIP-4444, this is a more radical method that limits the size of the historical blob to about 100 GB ((18 * 3600 * 24) * 128K * 6/12, assuming the block time = 12 seconds).
What are the consequences of deleting the historical data of all clients?The main problem is that the new node cannot synchronize to the latest state through the “Full Sync” mode. “Full Sync” is a data synchronization scheme that replaces historical data from the creation of the world to the latest block.Correspondingly, we must adopt the latest state of “Snap Snc” or “State Sync” to directly synchronize the Ethereum node.This method has been implemented in Geth and used as the default synchronization.
The historical data of the Ethereum main network delete the main network of Ethereum will also cause problems with Ethereum L2That is, the newly added Layer2 node cannot be synchronized to the current latest state by reloading all historical data of Layer2.In addition, because the L1 node does not maintain the L2 status, the “Snap Sync” method of L2 cannot directly generate the latest Layer2 state according to the Layer1 block, which violates the important assumptions of Layer2 inheritance of Ethereum security.
The estimated solution will rely on the third -party service of the Infmer / Etherscan / L2 project to store Layer2 historical data or status copy. This is a centralized solution implemented by the agreement outside and indirectly incentive.
The core question we want to explore is:
-
Can we find better decentralized solutions in storage and access?
-
Is it possible to find a solution that gives nodes directly incentive and get the Ethereum network itself (e.g., rely on the L1 contract)?
-
On all this, can we provide a completely decentralized and direct incentive solution to the Ethereum storage route?
Solution
Solution 1: Ethereum Portal Network
The Ethereum Portal network is a lightweight, decentralized network for connecting to the Ethereum protocol.It provides ETH_Call, Eth_getBlockbynumber and other Ethereum JSON-RPC interfaces. It converts the JSON-RPC request into a P2P request of a distributed hash table (DHT), similar to the IPFS network.Unlike IPFS that allows any data types and is easily affected by junk data, the Portal P2P network specializes in Ethereum data, such as historical block heads and transaction data, which is achieved through the light client verification technology built on the Portal network.
An important feature of the Portal network is.Its lightweight operation design and compatibility with limited equipment.It can run on nodes with several MB storage space and low memory, thereby promoting decentralization.Even mobile phones or Raspberry Pi devices may join the network to contribute to the Ethereum DA problem.
The development of the Portal network is consistent with the concept of the Ethereum client diversity. The client is written by Rust, JavaScript and NIM.The benchmark network and the historical network are available, and the status network is actively developing.It is worth noting that the Portal network does not provide direct incentives for data storage.
Figure: Portal network RUST client (Trin) with 100MB storage restrictions (TRIN) is running
Solution 2: Ethstorage network
ETHSTORAGE network is a decentralized incentive storage network that is specially used to store the EIP-4844 blob and receive funding from the ESP project.
InSmall trust:Unlike the existing solutions that require centralized data bridges, Ethstorage relies on the 1/M trust model of ETHSTORAGE nodes that do not need to be licensed by Ethereum.The process of storing blob is: users sign a transaction carrying Blob, calling the Put (key, blob_idx) method of storage contracts.Then, the storage contract will record the Blob hash on the chain.After that, the storage provider will download and store BLOB directly from the Ethereum DA network to bypass the data bridge.
InStorage costs are consistent with incentives:When the PUT () method is called, the transaction must send a storage fee (via msg.value) into the contract.After the storage node is submitted and verified the storage certificate under the successful chain, the storage fee gradually distributed to the storage node over time.Compared with the existing PROPOSER (Proposer) Ethereum storage fee model, which has a one -time storage fee, over time, the storage fee paid to the storage fee follows the discount cash flow model -assuming that over time, the storage storage is storage, storage, storage, storage, storage, storage, storage, storage, storage, storage, storage, storage, storage, storage.Costs will be reduced relative to ETH prices.This major innovation introduced by ETHSTORAGE has remained consistent with the contribution of the storage nodes in storage nodes.
InStorage proof:Storage proof is inspired by data availability sampling, and the sampling in Ethstorage is a blob saved for a period of time.In order to effectively verify the samples of the chain, ETHSTORAGE makes full use of smart contracts and the latest SNARK technology development.
InNo operation:Any storage node in ETHSTORAGE can be paid as long as the storage data is stored and the storage certificate is submitted regularly on the chain.
From the perspective of modular blockchain, ETHSTORAGE acts as Ethereum storage L2, but it charges the storage fee rather than a transaction fee.By indexing the Blob hash on the chain, Ethstorage is an Ethereum modular storage layer to improve the storage scalability and reduce costs (the target is about 1000 times).
In terms of development, ETHSTORAGE has integrated EIP-4844 on the Ethereum SEPOLIA testing network.We have conducted a pressure test of Ethstorage and Ethereum SEPOLIA test network, including writing about hundreds of GB of BLOB into Ethstorage.More than 100 community participants joined the Internet and successfully proved their local storage.
The main advantage of the ETHSTORAGE network is to provide decentralized direct incentives above Ethereum -as far as our current knowledge is concerned, this is a pioneering feature.However, the limitations of the network are designed for the fixed size BLOB.
ETHSTORAGE on the Ethereum SEPOLIA Testing Network
Look forward to the future
Although Ethereum Storage has not received the main attention, it is of great significance in the Ethereum ecosystem.With the rapid growth of the Ethereum network, the storage and accessability of Ethereum data have become a key challenge.The Portal network and Ethstorage network are still in the early stages, and there are many important long -term development directions to pay attention:
Decentalized low delay access Ethereum status data network:Accessing Ethereum status in a decentralized and verifiable manner is a key but challenging task.Using the traditional DHT network model, query account information usually requires multiple queries of internal Trie nodes stored in different P2P nodes.This often leads to a long period of delay.How to use the structure of the state tree to accelerate access is the key.The state network that the Portal network is soon launched is exactly the purpose of solving this problem.
Integration of Portal network and Ethstorage network:Portal network can seamlessly expand to support BLOB data.The Ethstorage team has partially implemented this feature.The next progress is to unify these networks and provide a decentralized JSON-RPC network that can be programmed programming through contracts.By combining the application logic in the contract with the large -scale BLOB storage provided by Ethstorage, we can enable new DAPPs on Ethereum, such as dynamic decentralized websites (such as decentralized Twitter/YouTube/Wikipedia, etc.).
The decentralized visit of the browser:Similar to the IPFS: // protocol of the data in the IPFS network, the web3 industry requires a native visit protocol for Ethereum to support the browser directly access to release the huge potential of Ethereum rich data.These data cover a wide range of areas, from tokens and account balances to NFT images and dynamic decentralized websites. All of them are achieved by the functions of smart contracts and future Ethereum storage.In this area, the Web3: // protocol defined by ERC-4804/6860 is currently actively developing and promoting to achieve this goal.
Advanced storage certificate of dynamic size data:In addition to fixed BLOB, exploring advanced storage certificates is also imperative for solving dynamic size data (such as historical blocks and even status objects, etc.).Developing complex algorithms can enhance the adaptability of the storage solution.
In our pursuit, we hope that through these efforts, we will contribute to the Ethereum route chart and lay the foundation for the decentralized storage solution of the Ethereum ecosystem in the future.
