jakiro
Author: Stanford BlockChain Club Source:W-Source Translation: Shan Ouba, Bitchain Vision Realm
introduction
Since its birth, blockchain and cryptocurrencies have been committed to changing the financial pattern by providing wider access permissions and removing intermediaries.Over time, the development of Web3 has expanded its application scenarios, emphasizing the potential of blockchain technology in the Internet of creating a creator prosperity and user control data.
While ensuring decentralization, the key part of the infrastructure of the empowerment of end users is to ensure that data is stored in a database with tough and anti -review.Although the centralized database is convenient and familiar, they fail to provide necessary security guarantee, and the permission of the database owner is required to limit the global application.
The distributed data storage system meets the demand for fault -tolerant and high tough storage by constructing a node network that stores, manage and share data.Remove the needs of central authority and distribute data in P2P to enhance security and transparency.The distributed storage system is usually based on the blockchain or similar technology, tending to copy data by redundant and availability.
Although the distributed storage system relies on unused storage capacity to provide security, data toughness, and potential cost benefits, they face challenges in terms of regulations and interoperability.
For truly open and accessible networks, distributed storage is essential.Any key aspect of any storage system is how to prove the way of storage and maintenance of data.For users, the more important issue is,How can data be proven to be stored and maintained?This is solved by data proof.
Overall, we have distinguished two types of proof:
-
Determine proof: Allowing data holders to create certificates for specific data to prove the hash value of its creation and matching data.This type of proof only publicly uses data as the hash for input generation.
-
Probability proof: Dependent probability to display the height of the basic data may be available.This is a proof of the degree of certain assumption that shows a specific hypothesis, which is suitable for data that has been released and can be retrieved when necessary.
-
Winning Post (winning post): Miners verify that they store data copy at a certain time, usually when the algorithm selects the miner to dig the next block.The short -term deadline ensures that they have data.
-
Window time and space certificate (WindowPost): A repetitive challenge, the miners submitted a certificate, indicating that they have maintained the data as required.The sealing data only when it is submitted to the miners is more expensive.
-
Data itself
-
Miners who seal data
-
Sealing time and date
-
The block height of the data sealing
-
Create a data block.
-
Disassemble the block data into a block of K × K to form a matrix.
-
Expand the matrix by adding a puppet-verified data to create a 2K × 2K matrix with Reed-Solomon coding.This type of encoding allows the entire data set from the subset of the data.
-
Calculate and combine independent Merkelgen per line and column of the extended matrix.
-
Finally, add all these combined Merkelen to the block data commitment in the block head to confirm the availability of the data.
The rest of this article will discuss the design selection of data storage and integrity in three different systems.The first is the Tagion focusing on high -capacity and scalable data, and then discusses how to ensure the large -scale storage of data from the decentralized storage network Filecoin.Finally, this article will explore CELESTIA focusing on storage and providing blockchain data.
Tagion
Architecture
Tagion is a decentralized network that is committed to high -volume transactions. The goal is to build a unique currency system based on technology and democratic governance.The project relies on innovative database architecture and encryption technology to achieve large -scale.It is not a blockchain, but a distributed ledger that optimizes storage with DART database.Tagion’s proof mechanism is an example of confirmation proof.
The core function of the DART database is as a distributed hash table to store data based on hash keys.With the increase of information storage, the structure will naturally generate more branches, and each branch supports as many as 256 combination files and sub -branches.
In addition to a distributed hash table, Tagion’s infrastructure can also be understood as sparse Murchel Tree (SMT).SMT is a key -value -based authentication data structure that supports standard database operations, such as finding, inserting, updating, and deleting.Each key value represents a leaf, and the fabricity of Merkelgen is sent to Merkelgen to send a family branch through recursively.
SMT verifies the existence of the element without access to unrelated data elements or downloading specific data fragments by allowing proofers to significantly improve efficiency.In addition, the independence of the median tree allows updates in any order without changing the final structure of the tree.
Tagion’s system uses the root hash, which contains all sub -branches, to quickly verify the data status with the smallest calculation.In order to further enhance the processing capacity, the system can create a sub -DART for a specific ecosystem, similar to the segment blockchain.These specified nodes manage data subsets, increase throughput, and enable the network to customize according to different applications, similar to the application chain.
Use DART to create a complete historical record that does not need to maintain system conversion.This means that data can be deleted, overall reduction of storage demand, and increasing the decentralization of systems through lightweight potentially.
Tagion uses HIBON (hash unchanged binary representative) to further promote the storage procedure to ensure that the data remains unchanged when entering, simplifying data retrieval based on associated hash.Hachs do not degenerate means that the data always generates the same hash when the data processing sequence is different.This is a verified technology that is used to accelerate data retrieval and writing in the database.
Through these mechanisms, TAGION not only securely stores data, but also efficiently verify its inclusion and integrity in the network.
Data integrity
All subsystems of Tagion will perform so -called random walking, check whether the data is stored and provided on demand.Nodes that fail to retain the verification challenge will be excluded from the network.
All archives include timestamps, which requires payment to extend the storage time.During the walking process, the system will check whether the payment is received. If not, delete the data and release the space.
Filecoin
Filecoin is a decentralized storage network that inspire miners to provide storage capacity through its native currency Filecoin.In order to earn these rewards, miners must generate proof to verify their storage capabilities.
The basic storage unit of Filecoin is called the sector, which has a standard size and a life cycle that the provider can extend.The design has carefully balanced between security and availability.All user data stored on Filecoin are encrypted, and multiple copies are distributed on the Internet to ensure that miners cannot access file content.
The influence of miners in the Filecoin network is directly proportional to its storage volume, which also allows them to participate in the consensus mechanism of the network.Filecoin virtual machine is responsible for performing smart contracts and promotes market operations, such as pairing storage providers and users.
Filecoin’s architecture is modular, allowing nodes to operate the specific part of the operating system as needed.For example, a node can only be used as a storage node without participating in market operations.
In order to ensure data integrity and availability, Filecoin relies on two algorithms: storage proof and replication proof.
Storage certificate
Miners in Filecoin generate proof to verify that they hold data copy at any given time.This proof can be achieved by challenges: the system can ask the miners to answer the correct answers correctly.
In order to ensure that the miners do not copy data when they propose challenges, and the challenge is designed to randomly target different parts of the data within an unpredictable interval.The combination of randomness and uncertainty of time interval makes the miners obtaining data is impossible, economical and irrational when they propose challenges.
Time and Space Certificate (POST)
Filecoin introduced the time and space certificate to ensure continuous storage and data availability.Time and space proves that by proposing an encryption challenge to the miners to verify the storage within the time interval.Only when the file is stored in the specified time frame, the miners can challenge.
Post includes two types of challenges:
Sealing is part of the copy of the certification algorithm, and the calculation dense type makes rational miners want to reduce the sealing demand as much as possible.
Copy certificate
Sealing is part of the copy of the certification algorithm, calculating dense types, and encouraging miners to reduce sealing frequency.Copy proof to ensure that users, miners created and stored the only copy on their physical hardware.This proof includes:
Miners are required to generate two proofs to provide users with security storage security storage, and only miners who provide actual storage can get rewards.Because it is proven that it is too large to be on the chain, the miners generate zero -knowledge simplicity and non -intestinal knowledge demonstration (ZKSNARK), and then submitted to the chain to make FileCoin the largest ZKSNARK user, with 6 million to 7 million certificates per day.
Overall,Filecoin combines the method of definitive proof (POREP) and probability proof (POST), which uses a hybrid method.
Celestia
The third example of this article is CELESTIA, a so -called data availability blockchain, which provides execution and data storage for modular blockchain, allowing them to outsourcing core functions.
With the rise of Ethereum Rollup, data usability solutions like Celestia are popular because it provides cheaper ROLLUP transaction data storage alternatives than Ethereum archive nodes.
Proof data usability
Unlike Filecoin, Celestia does not provide storage solutions for end users, but focuses on solving the problem of data usability.Data availability ensures that the blockchain data is correctly released.Generally, the blockchain node must download the entire block to verify the availability. This is a resource -intensive process that may hinder verification.
To simplify this process, Celestia uses data availability sampling (DAS).This method involves only a small portion of the data of light nodes until it reaches the predetermined level.If the data in the sample is available, the data is considered to be released, and the probability of data availability is proven.
The working principle is as follows:
In order to verify the availability, the light node randomly extracts the unique coordinates in the extension matrix, and then query the data block proved to the Merkel, which corresponds to these coordinates.If the response is correct, it means that the data of the entire block has a high probability that it is available.
The node then broadcasts the receiving data blocks received by the correct Merkel to the rest of the network.As long as the sampling is sufficient, the node can rebuild the entire block, so that Celestia can rely more on nodes with limited resources to verify, which helps decentralization.
When writing this article, Celestia is still new.However,Data availability sampling is a technology that may be adopted outside CELESTIA. Ethereum core developers are discussing adding it to the agreement to help expand.
in conclusion
In short, the various methods used to store and verify data availability in distributed networks are operating and actively used.
Tagion uses the DART database to increase throughput and support the development of special sub -ecosystems protected by sparse Murchel tree.
Filecoin’s architecture uses two different algorithms, time and space proof and replication proof, so that miners can verify and prove that they store data reliably.These proofs are subsequently recorded in the chain in the form of zero knowledge.
Celestia is used as a data availability layer, and the data block is expanded into a matrix using Reed-Solomon encoding.This structure allows light clients to perform random sampling to confirm the availability of data and bypass the need to download the entire data set.
With the continuous evolution of the distributed storage system, Tagion, Filecoin, and Celestia each proposed a unique strategy to ensure data integrity, availability and accessability.These platforms have made important contributions to the release and storage system that supports decentralized networks.
